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In this Issue 

if you thought that voltmeters, those ancient and fundamentaf instruments, 
had evolved about as far as possible and that there couldn't be much more 
to say about them, you'd be wrong. Today, they're usually called digital 
multimeters or DM Ms rather than voltmeters. You can find highly accurate 
■^P IIH^ ^^^^ ''^ calibration laboratories, very fast ones in automated test systems, 

^M^'9^^^ -^ . ^nd a whole spectrum of performance levels and applications in between 
Bll. ; ; these extremes. Generally, more speed means less resolution — that is. fewer 

^U T^^^^T digits Jn the measurement result. Conversely, a higher-resolution measure- 
HH ^^^^^^^^1 ment generally takes longer. Some DMMs are capable of a range of speeds 
and resolutions and allow the user to trade one for the other. The HP 3458A Digital Multimeter 
does this to an unprecedented degree. It can make 100,000 4V2-dlgit measurements per second 
or six 8y2-digit measurements per second, and allows the user an almost continuous selection 
of speed -versus-resolution trade-offs betv^een these limits. You'll find an introduction to the HP 
3458A on page 6. The basis of its performance is a state-of-the-art integrating analog-to-digital 
converter (ADC) that uses both multislope runup and muftislope rundown along with a two-Input 
structure to achieve both high speed and high precision (page 8), So precise is this ADC that it 
can function as a ratio transfer device for calibration purposes. With the ADC and a trio of built-in 
transfer standards, ail of the functions and ranges of the HP 3458A can be calibrated using only 
two external traceable standards^fOV and 10 kfl, The article on page 22 explains how this is 
possible. At the high end of its speed range, the ADC allows the HP 3458A to function as a 
high-speed digitizer, an unusual role for a DMM (page 39). In fact, ac voltage measurements are 
made by digitizing the input signal and computing its rms value, eliminating the analog rms-to-dc 
converters of older designs (page 1 5), Finally, moving data fast enough to keep up with the ADC 
was a design challenge in itself. How it was met with a combination of high-speed circuits and 
efficient firmware is detailed in the article on page 31, 

The seven papers on pages 50 to 90 are from the 1988 HP Software Engineering Productivity 
Conference and should be of interest to software engineers and users concerned with software 
defect prevention. Collectively, the papers spotlight areas where vigorous software engineering 
activity is occurring today, namely in structured and object-oriented analysis, design, and testing, 
and in the development of reliable metrics with which to measure software quality. ► In the paper 
on page 50, engineers from Yokogawa Hewlett-Packard and Tokyo University describe a joint 
effort to find the flaws in design procedures that increase the likelihood of human errors that result 
in program faults. Working backwards from faults to human errors to flawed procedures, they 
propose various structured analysis and design solutions to eliminate the flaws. ► The paper on 
page 67 urges expansion of the software defect data collection process so that project managers 
can not only determine how best to prevent future defects, but also build a case for making the 
necessary changes in procedures. The time required to collect and analyze the additional data 
is shown to be minimal. ► That complexity leads to defects is well-established, so monitoring the 
complexity of software modules during implementation should point out modules that will be defect 
prone. The paper on page 64 tells how HP's Waltham Division is taking this approach to improve 
the software development process, using McCabes cyclomatic complexity methc to measure 
complexity. ► Object-oriented programming, originally conceived for artificial intelligence applica- 
tions, is now finding wider acceptance. The paper on page 69 reports on problems and methods 
associated with testing software modules developed with an object-oriented language, C-^ ^ , for 
a clinical information system. > In the paper on page 75, Greg Kruger updates his June 1988 
paper on the use of a software reliability growth model at HP s Lake Stevens Instrument Division, 
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The model has generally been successful, but there are pitfalls to be avoided \n applying it. ► 
On a software project at HP's Logic Systems Division, some of the engineers used structured 
methods and some used unstructured methods. Was stnjctured design better? According to the 
paper on page 80, the results were mixed, but the structured methods offered enough benefits 
to justify their continued use. ► In software development, system analysis precedes design. The 
paper on page 86 descrfbes a new object-oriented approach suitable for analyzing today s increas* 
ingty larger and more complex systems. The authors believe that designs based on object-oriented 
specifications can be procedure-oriented of object-oriented with equal success. 

Modular measurement systems consist of instruments on cards that plug into a card cage or 
mainframe, A user can tailor a system to an application by plugging the right modules into the 
mafnframe. The VXIbus is a new industry standard for such systems. Modules and mainframes 
conforming to the VXIbus standard are compatible no matter what company manufactured them. 
The articles on pages 91 and 96 introduce the VXIbus and some new HP products that help 
manufacturers develop VXIbus modules more quickly. HP's own modular measurement system 
architecture conforms to the VXIbus standard where applicable. However, for high-performance 
RF and microwave instrumentation, HP has used a proprietary modular system interface bus 
(HP-MS IB). Patent rights to the HP-MSIB have now been assigned to the public so that this 
architecture can be used by everyone as the high-frequency counterpart of the VXIbus. 

R.P. Dolan 
Editor 



Cover 

So precise is the 3458 A Digital Multimeter that verifying some aspects of its performance is 
beyond the limits of conventional standards. In the HP Loveiand instrument Division Standards 
Laboratory, the HP 3458 As linearity is measured using a 10-volt Josephson junction array de- 
veloped by the U.S. National Institute of Standards and Technology. The array is in a specially 
magnetically shielded cryoprobe in the center of a liquid-helium-filled dewar {the lank with the 
protective "steering wheel") On lop of the dewar are a Gunn-diode signal source {72 GHz) and 
various microwave components. A waveguide and voltage and sense leads connect the array to 
the external components. For more details see page 24. 



What's Ahead 

Subjects to be covered in the June issue include: 

■ The HP 9000 Model 835 and HP 3000 Series 935 Midrange HP Precision Architecture 
Computers 

■ Programming with neurons 

■ A new 2D simulation model for etectromigration in thin metal films 

■ Data compression and blocking in the HP 7980XC Tape Dhve 

■ Design and applications of HP S702A Lightwave Component Analyzer systems 

■ A data base for real-time applications and environments 

■ A hardware/software tool to automate testing of software for the HP Vectra Personal 
Computer. 
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An 8y2-Digit Digital Multimeter Capable of 
100,000 Readings per Second and 
Two-Source Calibration 

A highly linear and extremely flexible analog-to-digital 
converter and a state-of-the-art design give this DMMnew 
performance and measurement capabilities for automated 
test, calibration laboratory, or R&D applications. 

by Scott D. Stever 



THE DIGITAL MULTIMETER OR DMM is among the 
mosl common and most versatile instruments avail- 
able for low-frequency and dc measurements in auto- 
mated tost, calibration laboratory, and bench RikD applica- 
tions. The use of general-purpose instrumentation in auto- 
mated measurement systems has steadily grown over the 
past decade. While early users of programmable instru- 
ments were elated to be able to automate costly, tedious, 
error- prone measurements or characterization processes, 
the sophistication and needs of today's users are orders of 
magnitude greater. The computing power of instrument 
controllers has increased manyfold since the mid-1970& 
and so have user expectations for the performance of mea- 
surement systems. Test efficiency in many applications is 
no longer limited by the device under test or the instrument 
controller's horsepower. Often either the configuration 
speed or the measurement speed of the test instrumentation 
has become the limiting factor for achieving greater test 
throughput. In many systems » the DMM is required to per- 
form hundreds of measurements and be capable of multiple 
tuoctions with various resolutiDns and accuracies. 

In same applications, several DMMs may be required to 
characterize a single device. For example, measurements 
requiring high precision may need a slower DMM with 
calibration laboratory performance. Usually, the majority 
of measurements can be satisfied by the faster, moderate- 
resolution capabilities of a traditional system DMM. In ex- 



treme cases* where speed or sample timing are critical to 
the application, a lower-resolution high-speed DMM may 
be required. A single digital multimeter capable of fulfilling 
this broad range of measurement capabilities can reduce 
system complexity and development costs. If it also pro- 
vides shorter reconfiguration time and increased measure- 
ment speed, test throughput can also be improved for au- 
tomated test applications. 

The HP 345«A Digital Multiineter (Fig. 1) was developed 
to address the increasing requirements for flexible, accu- 
rate, and cost-effective solutions in today's automated test 
applications. The product concept centers upon the syner- 
gistic application of state-of-the-art technologies to meet 
these needs. While it is tuned for high throughput in com- 
puter-aided testing, the HP 3458A also offers calibration 
laboratory accuracy in dc volts, ac volts, and resistance. 
Owners can trade speed for resolution, from 100,000 mea- 
surements per second with 4VLr-digil (IB-hit) resolution to 
six measurements per second with 8V2-digit resolution. At 
sy^-digit resolution, the DMM achieves 50,000 readings 
per second. To maximize the measurement speed for the 
resolution selected, the integration time is selectable from 
500 nanoseconds to one second in 100-ns steps. The effect 
is an almost continuoiis range of speed-versus-resolution 
trade-offs. 




Rg. 1 , The HP 3458 A Digltat Mui- 
umeter can make 100.000 4'/?- 
digit readings per second for high- 
speed automated test appllca- 
tfons. For cslibratton laboratory 
appfications. it can make six 8V^ 
digit readings per second- Fsne 
control of the integration aperture 
ailows a nearly continuous range 
of speed-versus-resouttion trade- 
offs. 
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: Capabilities 
Measurement ranges for the HP 34 58 A 's functions are: 

■ Voltage; 10 nV to lOOOV dc 

<1 mV to 700V rms ac 

■ Current: 1 pA to lA dc 

100 pA to lA rms ac 

■ Resistance: 10 ptH to 1 GO. 2-wire or 4-vkTre 

■ Frequenc^^: 1 Hz to 10 MHz 

■ Period: 100 ns to 1 s 

■ 16-bil digitizing at effective sample rates to 100 
megas am p 1 e s/secon d . 

The ac voltage bandwidth is 1 Hz to 10 MHz, either ac or 
dc coupled. 

To increase uptime, the HP 345 8 A is capable of two- 
source electronic calibration and serf-%^erifyingautocalibra- 
tion. Autocalibration enhances accuracy by eliminating 
drift errors with time and temperature. The dc voltage sta- 
bility is specified at eight parts per million over one year, 
or 4 ppm with the high-stability option. Linearity is speci- 
fied at 0.1 ppm, transfer accuracy at 0.1 ppm, and rms inter- 
nal noise at O.Gl ppm. Majcimum accuracies are 0,5 ppm 
for 24 hours in dc volts and 100 ppm in ac volts. Midrange 
resistance and direct current accuracies are 3 ppm and 10 



ppm. respectively. 

The HP 3450 A can transfer 16-bit readings to an HP 9000 
Series 200 or 300 Computer via the HP-IB (IEEE 438, lEC 
625] at 100,000 readings per second. It can change functions 
or ranges and deliver a measurement 200 times per second 
(over 300/s from the internal program memor%i, about four 
times faster than any earlier HP multimeter. 

The following five articles describe the technologies re- 
quired to achieve this performance and the benefits that 
result. In the first paper, the development of a single analog- 
to-digital converter capable of both high resolution and 
high speed is discussed. The second paper describes the 
development of a technique for the precise measurement 
of rms ac voltages. The application of these technologies 
to provide improved measurement accuracy over extended 
operating conditions and to provide complete calibration 
of the DMM from only two external traceable sources (lOV 
dc, 10 kll) is discussed in the third article. Hardware and 
firmware design to achieve increased measurement through- 
put is the topic of the fourth paper. The final paper dis- 
cusses several applicetions far the HP 3458A's ability lo 
perform high-resolution, high-speed digitizing. 
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An 8V2-Digit Integrating Analog-to-Digital 
Converter with 16-Bit, 100,000-Sample-per 
Second Performance 

This integrating-type ADC uses multi slope runup, 
multislope rundown, and a two-input structure to achieve 
the required speed, resolution, and linearity, 

by Wayne C. Goeke 



THE ANALOG-TO-D!G!TAL CONVERTER (ADC) 
design fortheHP3458ADigilal Multimeter was driv- 
en by die state-of-the-arl requirements for the system 
design. For example, autocalibration requirefi an ADC with 
Si/a-digit (28-bit) resolution and 71/2-digU (25-bit) integral 
linearity, and the digital ac techniqiie [see article ^ page 22) 
required an ADC capable of making 50,000 readings per 
second with IB-bit resolution. 

Integrating A DCs have always been known for their abil- 
ity to make high-resolution measurements, but tend to he 
relatively slow. When the HP 34 58 A 's design was started, 
the fastest integrating ADC known was in the HP 3456 A 
DMM. This ADC uses a technique known as multislope 
and is capable of making 330 readings per second. The HP 
3458 A "s ADC uses an enhanced implementation of the 
same multislope technique to achieve a range of speeds 
and resolutions never before achieved — from 16-bit resolu- 
tion at 100,000 readings per second to 28-bit resolution at 
six readings per second. In addition to high resolution, the 
ADC has high integral linearity — deviations are less than 
0.1 ppm (parts per million) of input. 

Multislope is a versatile ADC technique, allowing speed 
to be traded off for resolution within a single circuit. It Is 
easier to understand multislope by first understanding its 
predecessor, dual -si ope. 

Basic Dual-Slope Theory 

Dual-slope is a simple integrating- type ADC algorithm. 
Fig. 1 shows a simple circuit for implementing a dual -slope 
ADC. 

The algorithm starts with the integrator at zero volts. 
(This is achieved by shorting the integrator capacitor, C.) 
At time the unknown input voltage V^^^ is applied to the 
resistor R by closing switch SWl for a fixed length of time 
E^, This portion of the algorithm, in which the unknown 
input is being integrated, Is knowm as runup. At the end 
of runup [i.e., when SWl is opened], the output of the 
integrator, V„, can be shown to be 

v,,(t,]= -(i.^C] fVijtjdt 

OT» when Vj^ is time invariant » 



VoftJ= -(l/RC)V,,t,. 

Next a known reference voltage V^gf with polarity oppo- 
site to that of Vjn is connected to the same resistor R by 
closing SW2. A counter is started at this time and is stopped 
when the output of the integrator crosses through icero volts. 
This portion of the algorithm is known as rundown. The 
counter contents can be shown to be proportional to the 
unkimwn input. 

Vjy = VJtJ - (l/RC]V^ttd = 0. 

where t^ is the time required to complete rundown (i.e.. 
tf! - ^2 - t^). Solving for Vj,,, 

Letting N^ be the number of clock periods (T^^J during 



SWl 



Vin- 



Vfef- 



SW2 •— VW-4 



c 

HI- 



1^^ • Vo 




Runup 



Rundown 



Rg, 1. Duai'Siope mtegrating ADC (annlog-to^iigitBt con- 
verter) Circuit and a typical waveform 
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runup and N^ the number of dock periods during rundown, 
time cancels and 

The bfjaut>- of the dual-slope AE)C technique is its insen* 
silivity to the value of most of the circuit parameters. The 
values of R, C. and Tcl aU cancel from the final equation. 
Another advantage of the dual-slope ADC is that a single 
circuit can be designed to trade speed for resolution. If the 
runup time is shortened, the resolution will be reduced, 
but so will the time required to make the measurement. 

The problem with the dual-slope algorithm is that its 
resolution and speed are limited. The lime Tj„ for a dual- 
slope ADC to make a measurement is determined by: 

'^m = 2T^kM, 

where T^^ is the minimum theoretical time to make a full- 
scale measurement, T^^ is the period of the ADC clock, and 
M is the number of counts of resolution in a full-scale 
measurement. Even with a clock frequency of 20 MHz. to 
measure a signal with a resolution of 10,000 counts requires 
at least 1 millisecond. 

The resoliition of the dual-slope ADC Is limited fay the 
wideband circuit noise and the maximum voltage swing 
of the integrator, about ±10 volts. The wideband circuit 
noise limits how precisely the zero crossing can be deter- 
mined, rjetermining the zero crossing to much better than 
a millivolt becomes very difficull. Thus, dual-slope is lim- 
ited in a practical sense to four or five digits of resolution 
(i.e.. lOV/1 mV). 




Enhanced Dual-Slope 

The speed of the dual-slope *^VDC can be nearly doubled 
simply by using a pair of resistors, one for runup and the 
other for rundown, as shown in Fig. 2. 

The unknoT-vn voltage* V^j,, is connected to resistor Ry, 
which is much smaller than resistor R^* which is used 
during rundown. This allows the runup time to be short* 
ened by the ratio of the two resistors while maintaining 
the same resolution during rundown. The cost of the added 
speed is an additional resistor and a sensitivity to the ratio 
of the two resistors: 

Vi„ = - V^[Nd/NJlR„/Rd). 

Because resistor networks can be produced with excel- 
lent ratio tracking characteristics, the enhancement is ver>^ 
feasible. 

MultSslope Rundown 

Kn handed (iiial -slope reduces the time to perform runup. 
Multislope rundown reduces the time to perform rundown. 
Instead of using a single resistor (i.e,. a single slope) to 
seek zero, multislope uses several resistors [i,e., multiple 
slopes) and seeks zero several times, each time more pre- 
cisely. The ratio of one slope to another is a power of some 
number base, such as base 2 or base 10» 

Fig. 3 shows a multislope circuit using base 10. Four 
slopes are used in this circuit, with weights of 1000. 100» 
10. and T Each slope is given a name denoting its weight 



+Vref 



Vtn- 







Rb 1EK) 




C 



^^ 




Rurtdown 




Fig. 2. Entiance^ duBl-sfopB AOC Circuit uses two reststOfS. 
one for runup and one for rundown 



+S1000 +S10 -51 

Runiip Rundown 

Fig. 3. Base-W muimiope rundown cffcuit 
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and polarity. For example. Si 000 is a positive slope worth 
1000 counts per clock period and -SlOO is a negative 
slope worth - 100 counts per clock period. A slope Is con- 
sidered to be positive if it transfers ctiarge into the inte- 
grator. This may be confusing because the integrator [an 
inverting circuit) actuaily moves in a negative direction 
during a positive slope and vice versa. 

The mullislope rundown begins by switching on the 
steepest slope, SI 000. This slope remains on until the inte- 
grator output crosses zero, at which time it is turned off 
and the next slope, -SlOO, is turned on until the output 
crosses back through zero. The SlO slope follows next, and 
finally, the —Si slope. Each slope determines the inte- 
grator's zero crossing ten times more precisely than the 
previous slope. This can be viewed as a process in which 
each slope adds another digit of resol u t ion to the ru ndo w n . 

If each slope is turned off within one clock period of 
crossing zero, then each subsequent slope should take ten 
or fewer clock periods to cross zero. Theoretically, then^ 
the tune i^ to complete a multislope rundown is: 

where N is the number of slopes and B is the number base 
of the slope ratios. In practice, the time to complete run- 
down is higher, because it isn't always possible to to turn 
off each slope within a clock period of its y,mo crossing. 
Delays in detecting the zero crossings and delays in re- 
sponding by turning off the slopes cause the actual time 
to be: 



td < kNBT,i,. 



where k is a factor greater than one. The delay in turning 
off a slope results m the integrator output's overshooting 
zercj. For each clock period of overshoot, the following 
slope must take B clock periods to overcome the overshoot. 
Typical values of k range from two to four. The multislope 
rundown showm in Fign 3 completes a measurement yield- 
ing 10,000 counts of resolution in 4.0 /xs assuming a 20- 
MHz clock and k = 2. This is 12x5 times faster than the 
equivalent dual-slope rundown. 

Multislope can be optimized for even faster measure- 
ments by choosing the optimum base. Noting that the 
number of slopes, N, can be written as log|^[M). where M 
is the number of counts of resolution required from riin- 
dowrr, 

t,, < kBlogB(M)T,fc, 

This yields base e as the optimum base regardless of the 
required resolution. Using base e in the above example 



+Vret — o*N> — VvV^ 

SW1 ft,. 



-Vref 0'S> VsAr- 



4 



:n*^ 



results in o rundown time of 2.h ^s. This is a 60% increase 
in multislope rundown speed as a result of using base e 
instead of base 10* 

There is a cost associated with implementing multislope 
rundown. A resistor network must be produced with sev- 
eral resistors that have precise ratios. The tightest ratio 
tolerance is the reciprocal of the weight of tlie steepest 
slope and must be maintained to ensure linear ADC oper- 
ation. If the ratio tolerances are no tighter than 0.05%, then 
this requirement is feasible, IVlultislopealso requires a more 
complex circuit to control and accumulate the measure- 
menl, but with the reduced cost and increased density of 
digital circuits, this is also feasible. 

Multislope Runup 

Multislope runup is a modification of dual -slope runup 
with the purpose of increasing the resolulion of the ADC 
As mentioned earlier, the dual-slope technique's resolution 
is limited by the maximum voltage swdng of the integrator 
and the wideband circuit noise. Multislope runup allows 
the ADC to have an effective voltage swing much larger 
than the physical limitations of the integrator circuit 
hardware. 

The technique involves periodically adding and subtract- 
ing reference cliarge to or from the integrator during runup 
such that the charge from the unknown input plus the total 
reference charge is never large enough to saturate the inte- 
grator. By accounting for the total amount of reference 
charge tran.sf erred to the integrator during runup and add- 
ing this number to the result of rundown, a measurement 
can be made with much higher resolution. Fig, 4 shows a 
circuit for implementing multislope runup. 

A precise amount of reference charge is generated by 
applying either a positive reference voltage to resistor Rg 
or a negative reference voltage to resistor R^, for a fixed 
amount of time. The f ol lo w i ng table shows the foiu- possible 
runup reference currents using this circuit. 



Slope 


SVVa 


SVVb 


Integrator 


Curre 


Name 






Direction 




s. 


+v,,, 





Si 


-hi 


^ + 








- 





s_ 





-V..f 


y' 


-I 


S-0 


+v^, 


-V,,f 


- 






Fig. 4. Mutt! slope runup arcutL 



Notice that, like multislope rundown, S , adds charge to 
the integrator and S„ subtracts charge from the integrator. 
If we design the S, and S_ currents to have equal mag- 
nitudes that are slightly greater than that of the current 
generated by a full-scale input signal, then the reference 
currents will alw^ays be able to remove the charge ac- 
cumulating from tlie input signal. Therefore, the integrator 
can be kept from being saturated by periodically sensing 
the polarity of the integrator output and turning on either 
S . or S_ such that the integrator output is forced to move 
towards or across zero. 

Fig. n shows a typical multislope runup waveform. The 
dashed line shows the effective voltage swing, that is, the 
voltage swing without reference charge being put intn the 
integrator. The iotegrator output is staying within the limits 
of the circuit while the effective voltage swing ramps far 
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beyond the limit. The HP 3458A has an effective voltage 

swing of ±120,000 volts when making 8^'2-digit readings. 
which means the mndowB needs to resolve a millivoll to 
achieve an SvWigit reading (i.e.. 120.000V'0,OOlV ^ 
120,000.000 counts). 

The multislope runup algoritlim has two advantages over 
dual -slope runup: (IJ the runup can be continued for any 
length of time uithout saturating the integrator, and t2) 
resolution can be achieved during runup as well as during 
rundown. The HP 3458A resolves the first 4 Vz digits during 
runup and the final 4 digits during rundown to achieve an 
8V2-digit reading. 

An important requirement for any ADC is that it be linear. 
With the algorithm described above, multislope runup 
would not be linear. This is because each switch transition 
transfers an unpredictable amount of charge into the inte- 
grator during the rise and fall times. Fig. 6 shows tw^o 
waveforms that should result in the same amount of charge 
transferred to the integrator, but because of the different 
number of switch transitions, do not. 

This problem can be overcome if each switch is operated 
a constant number of limes for each reading, regardless of 
the input signah ff this is done, the charge transferred dur- 
ing the transitions will result in an offset in all readings. 
Offsets can be easily removed by applying a zero input 
periodically and subtracting the result from all subsequent 
readings. The zero measurement must be repeated period- 
ically because the rise and fall times of the switches drift 
with temperature and thus the offset wiil drift, 

Multislope runup algorithms can be implemented with 
constant numbers of switch transitions by alternately plac- 
ing an S^£, and an S_q between each runup slope. Fig. 7 
shows the four possible slope patterns between any two 
S^,j slopes. Varying input voltages will cause the algorithm 
to change between these four patterns ^ but regardless of 
which pattern is chosen, each switch makes one and only 
one transition between the first S^^ slope and theS ^ slope, 
and the opposite transition between the S_o slope and the 
second S+o slope. 

The cost of multislope runup is relatively small. The 
runup slopes can have the same weight as the first slope 
of multislope rundown. Therefore, only the opposite- polar- 



ity slope has to be added, along with the logic to implement 
the algorithm. 

MP 345dA ADC Design 

The design of the HP 3458A's ADC is based on these 
theories for a muhisbpe ADC. Decisions had to be made 
on how lo control the ADC. what number base to use. how 
fast the integrator can be slewed and remain linear, how 
much current to force into the integrator (i.e., the size of 
the resistors), and many other questions. The decisions 
were affected by Ijoth the high-speed goals and the high*res- 
olution goals. For example, very steep slopes are needed 
to achieve high speed, but steep slopes cause integrator 
circuits to behave too nonlinearly for high-resolution mea- 
surement performance. 

One of the easier decisions was to choose a number base 
for the ADC's multislope rundowm. Base e is the optimum 
to achieve the highest speed, but the task of accumulating 
an answer is difficult, requiring a conversion to binary. 
Base 2 and base 4 are both well-suited for binary systems 
and are close to base e. Base 2 and base 4 are actually 
equally fast, about 6% slower than base e, but base 2 uses 
twice as many slopes to achieve the same resolution. There- 
fore, base 4 was chosen to achieve the required speed with 
minimum hardware cost. 

Microprocessors have always been used to control mul- 
tislope ADCs, but the speed goals for the HP 3458A quickly 
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Fig. 5. Integrator output wa^etorrr} for multfstope runup. The 
dashed hne shows the effeaive ir^tegfator output ^oltagB 
swing. 



Fig, S» Ideally, these two wayeform^ would transfer equal 

charge into the integrator, but because of the different number 
of switch transitions, they do not 
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Fig. 7, Muftistope runup palter ns for an algorithm that keeps 
the number of swflch trar]srtions cor}stant. 



elimmated the possibility of using a microprocessor to con- 
trol the ADC algorithm. It was anticipated that the AFJC 
clock frequency would have to be between 10 and 20 MHz, 
and making decisions at these rates requires dedicated 
hardware. Therefore, a gate array was chosen to implement 
state mai:hiiieji running a! 20 MHz for ADC control. The 
ADC conlrol and accumulator functions consume approx- 
imately half of a 6000-gate CMOS gate array. The other half 



of the gate array is devoted to the timing and counting of 
triggers and a UART (universal asynchronous receiver/ 
transmitter) to transfer data and commands through a 2- 
Mbit/s fiber optic link to and from the ground-referenced 
logic (see article, page 31). 

The number of slopes and the magnitude of the currents 
far each slope are more subtle decisions. If the slope cur- 
rents get too large, they stress the output stage of the inte- 
grator's operational amplifier, which can cau,se nonlinear 
behavior. If the currents are too small, switch and ampiifier 
leakage currents can become larger than the smallest slope 
current, and the slope current would not be able to converge 
the integrator toward zero. A minimum of a microampere 
for the smallest slope was set to avoid leakage current prob- 
lems. Also* it was believed that the integrator could handle 
several milliamperes of input current and remain linear 
over five or six digits » but that less than a milliampere of 
input current would be required to achieve linearity over 
seven or eight digits. On the other hand, greater than a 
milliampere of current was needed to achieve the high- 
speed reading rate goal. Therefore, a two-input ADC struc- 
ture was chosen. 

As shown in Fig. 8, when making high-speed measure- 
ments, the input voltage is applied through a 10-kfi resistor, 
and the ADC's largest slopes ^ having currents greater than 
a milliampere, are used. When making high-resolution 
measurements, the input voltage isapplied through a 50'kil 
resistor and the largest slopes used have less than a milliam- 
pere of current. The largest slope vvas chosen to be Si 024, 
having 1,2 fxA of current. This led to a total of six slopes 
(S1024, S25fj, S64, Slfi. S4, and Si) with Si having about 
1.2 ^ A of current. Si 024 and S256 are bulb used during 
mult i slope runup; therefore, both polarities exist for both 
slopes. The ±S256 slopes (0,3 mA) are used when the 
50-kohm input is used and both the ±51024 and the ±8256 
slopes [1.5 mA total) are used In parallel when the lO-kli 
input is used. The S256 slope is 25% stronger than a full- 
scale input to the 50-kil resistor, which allow^s it to keep 
the integrator from saturating, The lO-kQ input is five times 
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Fig. 8- Simpfiffed HP 345&A ADC 
Circuit. 
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stronger than the BO-kohm input: thus, by using both SI 02 4 
and S256, 25% stronger reference slopes can be maintained 
during high -speed measurements. 

Integrator 

The integrator's operational amplifier behaves more non* 
linearly as the integrator slew rate (i.e., the steepest slope] 
approaches the slew rate of the amplifier. Two factors de- 
termine the integrator slew rate: the total current into the 
integrator and the size of the integrator capacitor. Wanting 
to keep the integrator slew rate less than 10V//its led to an 
integrator capacitor of 330 pF. This capacitor must have a 
very small amount of dielectric absorptian since 50 IC of 
charge is one count. 

The integrator circuit has to respond to a change in refer- 
ence current and settle to near 0,01% before the next pos- 
sible switch transition (about 200 ns). It also has to have 
low' voltage and current noise, about lOOVjiis slew rate, a 
dc gain of at least 25,000, an offset voltage less than 5 mV, 
and a bias current of less than 10 nA. A custom amplifier 
design was necessary to achieve all the specifications. 

Resistor Network 

The resistor network has several requirements. The most 
stringent is to obtain the lowest ratio-tracking temperature 
coefficient possible. It is important to keep this coefficient 
low becau.se the gain of the ADC is dependent an the ratio 
of the input resistor to the mnup reference slope resistors* 
An overall temperature coefficient of 0.4 ppmrc was 
achieved for the ADC. Even at this level, a temperature 
change of 0.1 "C results in a five-count change in a full-scale 
BV2-digil measurement. [Autotiafibration increases the gain 
stability to greater than 0.15 ppm/^C.) 

Another requirement for the resistor network is to have 
a low enough absolute temperature coefficient that non- 
linearities are not introduced by the self-heating of the 
resistors. For example^ the 50-kil input resistor has a input 
voltage that ranges from + 12V to - 12V. There is a 2.88- 
milliwatt power difference between a OV input and a 12V 
input. If this power difference causes the resistor to change 
its value, the result is a nonlinearity in the ADC. A 0.01 Ti 
temperature change in a resistor that has an absolute tem- 
perature coefficient of 1 ppm/°C causes a one-count error 
in an a^/:!-digit measurement. The network used in the HP 
3458A's ADC shows no measurable scif-heating non- 
linearities, 

The final requirement of the resistor network is that it 
maintain the ratios of the six slopes throughout the life of 
the HP3458A. The tightest ratio tolerance is approximately 
0,1% and is required to maintain linearity of the high-speed 
measurements. This is a relatively easy requirement. To 
maintain the ADC's 8'/i-digit differential linearity at less 
than 0.02 ppm requires ratio tolerances of only 3%. 

Switches 

A last major concern for the ADC design was the switches 
required to control the inputs and the slopes. Because the 
switches are in series with the resistors, they can add to 
the temperature coefficient of the ADC. A custom chip 
design was chosen so that each switch could be scaled to 
the size of the resistor to which it is connected. This allows 



the ADC to be sensitii^e to the ratio-tracking temperature 
coefficient of the switches and not to the absolute temper- 
ature coefficient. Another advantage of the custom design 
is that it allo%vs the control signals to be latched just before 
the drives to the switches. This resynchionizes the signal 
with the clock and reduces the timing jitter in the switch 
transitions. The result is a reduction m the noise of the 
ADC. 

Performance 

The performance of an ADC is limited by several non- 
ideal behaviors. Often the stated resolution of an ADC is 
limited by differential linearity or noise ev^en though the 
number of counts generated would indicate much finer 
resolution. For example, the HP 3458A's ADC generates 
more than 914 digits of counts but is only rated at QVz digits 
because the ninth digit is very noisy and the differential 
linearity is about one eight-digit count. Therefore, w^hen 
stating an ADC's speed and resolution, it is Important to 
specify under w^hat conditions the parameters are valid. 
Fig. 9 shows the speed-versus-resolution relationship of 
the HP 345 8A ADC assuming less than one count of rms 
noise. 

Given a noise level, there is a theoretical limit to the 
resolution of an ADC for a given speed. 11 can be shown 
that the white noise bandwidth of a sigurtl that is the output 
of an integration over time T is 
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BW = 1/2T, 

If rundown took ;cero time then an integrating ADC could 
sample once every T seconds. At this rate, the counts of 
resolution, M, of an ADC are noise-limited to 

where Vfg is the full-scale input voltagejo the ADC and V^, 
is the white noise of the ADC in V/Viiz. Fig. 9 shows the 
hest theoretical resolutioji for an ADC with rms noisti of 
100 nV/Vfe and a full-scale input of 10 voils. The HP 
3458A comes very close to the theoretical limit of an ADC 
with a white noise of 130 nV/VHz near the 7-dLgit resolu- 
tion region. At lower resolutions the ADC's rundown time 
becomes a more significant portion of the overall measure- 
ment time and therefore pulls I he ADC away from the 
theoretical limit. At higher resolutions the 1/f noise of the 
ADC forces a measurement of zero several timers within 
the measurement cycle to reduce the noise to the SV^-digit 
level This also reduces the measurement speed. 

Another way of viewing the ADC's performance is to 
plot resolution versus aperture. The aperture is the integra- 
tion time, that is, the length of runup. This is shown in 
Fig. 10 along with tlie 100-nV/v15z noise limit and the 
ADC"s resolution without regard to noise. At smaller aper- 
tures, the HP 345 8 A 's resolution is less tha[i the theoretical 
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Fig. 10. HP 3456 A ADC aperture (runup bme) versus fesolu- 



noise limit hecause it is limited by noise in detecting the 
final zero of rundown. That is, the algorithm does not have 
enough resolution to achieve the theoretical resolution. 

Linearity 

High -resolution linearity was one of the major challenges 
of the AlJf* design. The autocalibration technique requires 
an integrtil linearity of OJ ppm and an differential linearity 
of 0.02 ppm. One of the more significant problems was 
verifying the integral linearity^ The most linear E:ommf!r- 
cially available device we could find was a Kelvin-Vtirley 
divider, and its best specificatitjn was 0.1 ppm of input. 
Fig. 11 compares this with the ADCTs requirements, show- 
ing that it is not adequate. 

Using low-thermal-EMF switches, any even-ordered de- 
viations from an ideal straight line can be detected by doing 
a turnover lest. A turnover test consists of three steps: (1) 
measure and remove any offset, [21 measure a vol I age, and 
(31 switch the polarity of the voltage (i,e., turn the voitage 
over) and re measure it. Any even -order errors will produce 
a difference in the magnitude of the two nonzero voltages 
measured. Measurements of this type can be made to within 
0.01 ppm of a tOV signal, This left us with only the odd- 
order errors to detect. Fortunately, the U.S. National Bureau 
of Standards had developed a loseiilison junction array 
capable of generating voltages from - lOV to 4- lOV. Using 
a lOV array we were able to measure both even -order and 
orhj -order errors with confidence to a few hundredths of 
a ppm, Fig. 4a on page 23 shows the integral linearity error 
of an HP 34^8 A measured usinga fosephson junction array. 

The differenlial linearity can be best seen by looking at 
a small interval about zero volts. Here a variable source 
need only be linear within 1 ppm on its 100-mV range to 
produce an output that Is within 0.01 ppm of 10 volts. Fig. 
4b on page 23 shows the differential linearity of an HP 
3458A. 
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Precision AC Voltage Measurements Using 
Digital Sampling Techniques 

instead of traditional DMM techniques such as thermal 
conversion or analog computation, the HP 3458 A DMM 
measures rms ac voltages by sampling the input signal and 
computing the rms value digitally in real time. Track-and- 
hold circuit performance is critical to the accuracy of the 
method. 

by Ronald L. Swerlein 



THE HP 3458A DIGITAL MULTIMETER implements 
a digital method for the precise meusuremt^ni of rms 
ac voltages. A technique simiiar to that of a modern 
digitizing oscilloscope is used to sample the input vohage 
waveform. The rms value of the data is computed in real 
time lo produce the final measurement result- The HP 
345 8A objectives for high-precision digital ac measure- 
ments required the development of both new measurement 
algorithms and a track-and-hold circuit capable of fulfilling 
these needs. 

Limitations of Conventional Techniques 

All methods for making ac rms measurenients tend lo 
have various performancB Hraitations. Depending on the 
needs of the measurement, these limitations take on differ- 
ent levels of importance. 

Perhaps the most basic specification of performance is 
accuracy. For ac measurements, accuracy has lo be 
specified over a frequency band. Usually, the best accuracy 
is for sine waves at midband frequencies [typically 1 kRz 
to 20 khiz). Low-frequency accuracy usually refers to fre- 
quencies below 200 Hz (some techniques can work down 
to 1 H^), Bandwidth is a measure of the technique's perfor- 
mance a I higher frequencies. 

Linearity is another measure of accuracy. Lineajity is a 
measure of how much the measurement accuracy changes 
when the applied signal voltage changes, hi general linear- 
ity is a function of frequency just as accuracy is, and can 



be included in the accuracy specifications. For instance, 
tiie accuracy at 1 kH^ may be specified as 0.02% of reading 
+ 0,01% of range while the accuracy at 100 kHz may be 
specified as 0.1% of reading + 0.1% of range. The percent*- 
of-range part of the specification is where most of I he linear- 
ity error is found. 

If a nonsinusoid is being measured, most ac rms measure- 
ment techniques exhibit additional error; Crest-factor error 
is one way to characterize this performance. Crest factor 
is defined as the ratio of the peak value of a waveform to 
its rms value. For example, a sine wave has a crest factor 
of 1.4 and a pulse train wath a duty cycle of 1/25 has a 
crest factor of 5, Even whan crest factor error Is specified, 
one should use caution when applying this additional error 
if it Is not given as a function of frequency. A signal with 
a moderately high crest factor may have significant fre- 
quency components at 40,000 times the fundamental fre- 
quency. Thus crest factor error should be coupled with 
bandwidth information in estimating the accuracy of a mea- 
surement. In some ac voltmeters, crest factor specifications 
mean only that the voltmeter's internal amplifiers will re- 
main unsaturated with this signal, and the accuracy for 
nonsinusoids may actually be unspecified. 

Some of the secondary performance specifications for 
rms measurements are short-term reading stability, settling 
time, and reading rate. These parameters may have primary 
importance, however, depending on the need of the mea- 
surenient. Short-term stability is self-explanatory, but the 
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difference between settling time and reading rate is some- 
times confusing. Settling time is usually specified as the 
time that one should wait afterafulUscale signal amplitude 
change before accepting a reading as having full accuracy. 
Reading rate is the rate at which readings can be taken. Its 
possible for an ac voltmeter that has a one-second settling 
time to be able to take more than 300 readings per second. 
Of course J after a fulUscaJe signal swing, the next 299 read- 
ings would have degraded accuracy. But if the input signal 
swing is smaller than full-scale, the settling time to 
specified accuracy is faster. Therefore, in some situations, 
the 300 readings/second capability is actually useful even 
though the settling time is one second. 

The traditional methods for measuring ac rms voltage 
are thermal conversion and analog computation. The basis 
of thermal conversion is that the heat generated in a resis- 
tive element is proportional to the square of the rms voltage 
applied to the element. A thermocouple is used to measure 
this generated heat. Thermal conversion can be highly ac- 
curate with both sine waves and waveforms of higher crest 
factor. Indeed, this accuracy is part of the reason that the 
U.S. National Institute of Standards and Technolog>' (for- 
merly the National Bureau of Standards) uses this method 
to supply ac voltage Iraceability. It can also be used at very 
high frequencies (in the hundreds of MHz]. But thermal 
conversion tends to be slow (near one minute per reading) 
and tends to exhibit degraded performance at low frequen- 
cies (below 20 Hz]. The other major limitation of thermal 
conversion is dynamic range, Low output voltage, low ther- 
mal coupling to the ambient environment, and other factors 
limit tbis tecbnique to a dynamic range of around 10 dB. 
This compares to the greater than 20 dB range typical of 
other techniques. 

Analog computation is the other common technology 
used for rms measurements. Essentiaily, the analog con- 
verter uses logging and antilogging circuitry to implement 
an analog computer that calcuhites the squares and square 
roots involved in an nns ni ea.su rement. Since the rms av- 
eraging is implemented using electronic filters (instead of 
the physical thermal mass of the thermal converter], analog 
computation is very flexible in terms of reading rate. This 
flexibility is the reason that this technique is offered in the 
HP 3458 A Multimeter as an ATE-optimized ac measure- 
ment function (SETACV ANA]. Switchable fitters offer set- 
tling times as fast as 0.01 second for frequencies above 10 
kHz. With such a filter, reading rates up to 1000 readings/ 
second may be useful. 

Analog computation does have some severe accuracy 
draw^backs, however. It can be very accurate in the midband 
audio range, but both its accuracy and its linearity tend to 
suffer severe degradations at higher frequencies. Also, the 
emitter resistances of the transistors commonly used to 
implement the logging and antilogging functions tend to 
cause errors that are crest-factor dependent. 

Digital AC Technique 

Digital ac is another way to measure the rms value of a 
signal. The signal is sampled by an analog-to-digital con- 
verter (ADC) at greater than the signal's Nyquist rate lo 
avoid aliasing errors, A digital computer is then used to 
compute the rms value of the applied signal. Digital ac can 



exhibit excellent linearity that doesn*t degrade at high fre- 
quencies as analog ac computation does. Accuracy with 
all waveforms is comparable to thermal techniques without 
their long settling times. It is possible to measure low fre- 
quencies faster and with better accuracy than other methods 
using digital ac measurements. Also^ the technique lends 
itself to autocalibrati on of both gain and frequency response 
errors using only an external dc voltage standard {see arti- 
cle, page 22). 

In its basic form, a digital rms voltmeter would sample 
the input w^aveform with an ADC at a fast enough rate to 
avoid aliasing errors. The sampled voltage points (in the 
form of digital data) would then be operated on by an rms 
estimation algorithm. One example is shown below: 

Num = number of digital samples 

Sum = sumof digital data 

Sumsq = sum of squares of digital data 

ac rms - SQ R( [ Su msq - S um -^ 3 u m N u m ), N u m ) 

Conceptually, digital rms estimation has many potential 
advantages that can be exploited in a digital multimeter 
(DMM). Accuracy J linearity over the measurement ranges 
frequency response, short-term reading stability, and crest 
factor performance can all be excellent and limited only 
by the errors of the ADC, The performance limitations of 
digital ac are unkocnvn at the present time since ADC tech- 
nology is continually improving. 

Reading rates can be as fast as theoretically possible be- 
cause ideal averaging filters can be implemented in 
firmw^are, Law-1'requency settling time can be improved by 
measuring the period of the input waveform and sampling 
only over integral numbers of periods. This wrould allow 
a i-Hz waveform to be measured In only two seconds — one 
second to measure the period and one second to sample 
the wciveform. 

Synchronous SubsampHng 

A thermal converter can measure ac voltages in the fre- 
quency band of 20 Hz to 10 MHz with state-of-the-art accu- 
racy. Sampling rates near 50 MHz are required to measure 
these same frequencies digitally, but present A DCs that can 
sample at this rate have far less linearity and stability than 
is required for state-of-the-art accuracy in the audio band. 
If the restriction is made that the signal being measured 
must be rep eti live ^ however, a track-and-hold circuit can 
be used ahead of a slower ADC with higher stability to 
create an ADC that can effectively sample at a much higher 
rate. The terms "effective time sampling," "equivalent time 
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sampling/* and '^subsampling" are used iiitercliangeabiy 
to describe this technique. 

The concept of subsampling is used by digitizing oscil' 
loscopes to extend their sample rate to %vell beyond the 
intrinsic speed of the ADC. The concept is to use a trigger 
level circuit to establish a time reference relative to a point 
on a repetitive input signal. A timing circuit, or time base, 
is used to select sample delays from this reference point 
in increments determined by the required effective sam- 
pling rate. For example, movingtlie sampling point in 10-ns 
increments corresponds to an effective sampling rate of 
100 MHz. A block diagram of a subsampling ac voltmeter 
IS shown in Fig, 1. 

Fig. 2 is a simple graphic example of subsampling. Here 
we have an ADC that can sample at the rate of five samples 
for one period of the waveform being measured. We want 
to sample one period of this waveform at an effective rate 
of 20 samples per period. First, the timing circuit waits for 
a positive zero crossing and then takes a burst of five read- 
ings at its fastest sample rate. This is shown as "First Pass" 
in Fig, 2. On a subsequent positive slope, the time base 
delays an amount of time equal to one fourth of the ADC's 
minimum time between samples and again takes a burst 
of five readings. This is shown as *' Second Pass," This 
continues through the fourth pass, at which time the 
applied repetitive waveform has been equivalent lime sam- 
pled as if the ADC could acquire data at a rate four times 
faster than it actually can. 

The digital ac measurement technique of the HP 345aA 
is optimized for precision calibration laboratory measure- 
ments. Short-term measurement stability better than 1 ppm 
has been demonstrated. Absolute accuracy better than 100 
ppm has been shown. This accuracy is achieved by auto- 
matin internal adjustment relative to an external lOV dc 
standard. No ac source is required (see article, page 22]. 
The internal adjustments have the added benefit of provid- 
ing a quickp Independent check of the voltage ratios and 
transfers that are typically performed in a standards labo- 
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Fig- 2. An example of subsamptrng 



ratory every day. Fast, accurate 1-Hz measurements and 
superb performance with nonsinusoids allow calibration 
laboratories to make measurements easily that were previ- 
ously ver>^ difficult. 

The HP 3458 A enters into the synchronously subsampled 
ac mode through the command SETACV SYNC, For optimal 
sampling of the input signal, one must determine the period 
of the signal, the number of samples required* and the 
signal bandwidth. The measurement resolution desired 
and the potential bandwidth of the input waveform are 
described using the commands RES and ACSAfrJD, The 
period of the input signal is measured by the instrument. 
The more the HP 3458 A knows about the bandwidth of 
the input and the required measurement resolution, the 
better the job it can do of optimizing accuracy and reading 
rate. Default values are assumed if the user chooses not to 
enter more complete information. An ac measurement 
using the SYNC mode appears to function almost exactly 
the same to the user as one made using the more conven- 
tional analog mode. 

Subsampfed AC Algorithm 

The algorithm applii^d internallybytheHP3458A during 
each st][)sampled ac measurement is totally invisible to the 
user. The first part of a subsampled ac measurement is 
autolevel. The input waveform is randomly sampled for a 
peri^od of time long enough to get an idea of iU minimum 
and maximum voltage points. This time is at least tme cycle 
of the lowest expected frequency value (the low-frequeucy 
value of ACBAND). The trigger level is then set to a point 
midway between the minimum and maximum voltages, a 
good triggering point for most waveforms. In the unlikely 
event that this triggering point does not generate a reliahle 
trigger, provision is made for the user to generate a trigger 
signal and apply it to an external trigger input. An example 
of such a waveform is a video signal, Even though video 
signals can be repetitive, they are difficult to trigger on 
correctly with just a standard trigger level. 

With the trigger level determined,! he period of the input 
waveform is measured. The measured period is used along 
with the global parameter RES to determine subsampling 
parameters. These parameters are used by the timing cir- 
cuitry in the HP 345BA to select the effective sample rate, 
the number of samples, and the order in which these sam- 
ples are to be taken. In general, the HP 3458A tries to 
sample at the highest effective sample rate consistent with 
meeting the twin c:nnstrainls of subsampling over an inte- 
gral number of input wavefarm periods and restricting the 
total number of samples to a minimum value large enough 
to meet the specified resolution. This pushes the frequency 
where aliasing may occur as high as possible and also per- 
forms the best rms measurement of arbitrary waveforms of 
high crest factor. The number of samples taken will lie 
somewhere between 4/RES and 6/RES depending on the 
measured period of the input waveform. 

The final step is to acquire samples. As samples are taken, 
the data is processed in real time at a rate of up to 50,000 
samples per second to compute a sum of the readings 
squared and a sum of the readings. After all the samples 
are taken, the two sum registers are used to determioH 
standard deviation [ACV fuorition), or rms value (ACDCV 
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function). For example, suppose a 1-kJIz waveform is being 
measured and the specified measurement resolution is 
0,001%. When triggered, the HP 3458 A will take 400.000 
samples using an effective sample rate of 100 MHz, The 
timing circuit waits for a positive-slope trigger level. Then, 
after a small fixed delay, it takes a burst of 200 readings 
spaced 20 ^s apart. It wait?* for another trigger, and when 
this occurs the timing circuit adds 10 ns to the previous 
delay before starting another hurst of 200 readings. This is 
repeated 2,000 times, generating 400,000 samples. Effec- 
tively, four periods of the 1-kIiz signal are sampled with 
samples placed every 10 ns. 

Sources of Error 

Internal time base jitter and trigger jitter during subsam- 
pling contribute measurement uncertainty to the rms mea- 
surement. The magnitude of this uncertainty depends on 
the magnitude of these timing errors. The internal time 
base jitter in the HP 3458A is less than 100 ps rms. Trigger 
jitter is dependent on the input signal's amplitude and 
frequency, since internal noise will create greater time un- 
certainties for slow-slew-rate signals than for faster ones. 
A readily achievable trigger jitter is 100 ps rms for a 1-MHz 
input. Fig. 3 is a plot generated by mathematical modeling 
of the performance of a 400,000-iiample ac measurement 
using the HP 3458A\s subsampling algorithm (RES = 
0.001%] in the presence of 100-ps time base and trigger 
jitters. The modeled errors suggest the possibility of stable 
and accurate ac measurements with better than 6-digJt ac- 
curacy. 

Errors other than time jitter and trigger jitter limit the 
typtcal absolute accuracy of the HP 3458A to about 50 ppm, 
but there is reason to believe that short-term stability is 
better than 1 ppm. Many five-minute stability tests using 
a Datron 4200 AC Calibrator show reading-to-reading stan- 
dard deviations betw^een 0,7 ppm and 3 ppm. Other mea- 
surements independently show^ the Datron 4200 to have 
similar short-term stability, More recently, tests performed 
using a Fluke 5700 Calibrator, which uses a theoretically 
quieter leveling loop, show standard deviations under 0.6 
ppm. 

The above algorithm tries to sample the applied signal 
over an integral number of periods. To do this, the period 
of the signal must first be measured. Krrors in measuring 
the period of the input waveform %vill cause the subsequent 
sampling to cover more or less than an integral number of 
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periods. Thus, the accuracy of the subsampled ac rms mea- 
surement is directly related to how accurately the period 
of the input waveform is known relative to the internal 
.sample time base clock. 

Period measurements in the HP 3458A are performed 
using reciprocal frequency counting techniques. This 
method allows accuracy to be traded off for measurement 
speed by selecting different gate times. A shorter gate time 
contributes to a faster measurements but the lower accuracy 
of the period determination contributes to a less accurate 
ac measurement. Fig. 4 is a graph of the error introduced 
into the mis measurement by various gate times. At high 
frequencies, this error is a constant dependent on the res- 
olution of the frequency counter for a given gate time. At 
low^er frequencies, trigger time jitter increases, causing in- 
creased error, because random noise has a larger effect on 
slowrer signals. At still lower frequencies, where the period 
being measured is longer than the selected gate time, this 
error becomes constant again. This is because the gate time 
is always at least one period in length, and as the frequency 
is lowered, the gale time increases just fast enough to cancel 
the effect of increasing trigger jitter. 

Any violation of the restriction that the input w^aveform 
be re[3etitive will also lead to errors. A common condition 
is amplitude and frequency modulation of the input. If this 
modulation is of a fairly small magnitude and is fast com- 
pared to the total measurement time this violation of the 
repetitive requirement will probably be negligible. At most, 
reading4o-reading variation might increa.se slightly. If 
these modulatiuns become large, however, subsampled ac 
accuracy can be seriously compromised. The signal sources 
typically present on a lab bench or In a calibration labora- 
tory work quite well with the subsampling algorilhm of 
the HP 3458A. 

Random noise spikes superimposed on an input can 
make an otherwise repetitive input waveform appear non- 
repetitive. Induced current caused by motors and electrical 
devices turning on and off is just one of many ways to 
generate such spikes. Large test systems tend to generate 
more of this than bench and calibration laboratory environ- 
ments. Low- voltage input signals (below 100 inV) at low 
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frequencies are the signals most susceptible to these errors. 

T%%'o ways are provided by the HP 345 8 A to deal with 
these potential errors. The first is to use the internal 80 -kHz 
low-pass trigger filter to reduce high-frequencj^ trigger 
noise (LFFLTEH OK). If this is not enough, provision is made 
for accepting external synchronizatioji pulses. In principle, 
getting sybsampled ac to 'ivork in a noisy environment is 
no more difficult than getting a frequency counter to work 
in the same environment. 

IS nonsinusoidal signals are being measured, the subsam- 
pling algorithm has some addiUonal random errors that 
become greater for signals of large crest factor. All noji- 
sinusoidal repetitive signals have some of their spectral 
energy' at frequencies higher than their fundamental fre- 
quency. Signals of high crest factor generally have more of 
this high-frequency energy than those of lower crest factor. 
Random timing j I Iter, which tends to affect higher frequen- 
cies the most, will create measurement errors that are 
greater for large-crest-factor signals. These random errors 
can be reduced by specifying a higher-resolution measure- 
ment, which forces more samples per reading to be ac- 
quired. The additional measurement error induced by a 
signal crest factor of 5 can be as low as 50 ppm in the HP 
3458A. 

Trac k-a n d - H o td C i rcu it 

Track-aiul-hold performance is critical to the accuracy 
of digital ac measurements. Track-and-hold linearity, 
bandwidth, frequency flatness, and aperture jitter all afiect 
the error in a sampled measurement. To meet the HP 3458 A 
performance objectives » track-and-hold frequency response 
flatness of ±0.001^% [15 ppm) was required from dc to 50 
kHz, along with a 3-d B bandwidth of 15 MHz, In addition, 
Iti-bit Hneanty below 50 kHz and low aperture jitter were 
needed. A custom track-and-hold amfdlfier was developed 
to meet these requiremenls, 

The most basic implementation of a track-and-hold cir- 
cuit^-a switch and a capacitor — is shown in Fig. 5. If the 
assumption is made that the switch is perfect (when open 
it has infinite impedance and when closed it has zero im- 
pedance) and if it assumed that the capacitor is perfect (ni> 
dielectric absorption), then this is a perfect track-and-hold 
circuit. The voltage on the capacitor will track the inpul 
signal perfectly in track mode, ajid when the switch is 
opened, the capacitor will hold its value milil the switch 
is closed. Atso» as long as the buffer amplifier's input im- 
pedance is high imd well-behaved, its bandwidth can be 
much lower than the bandwidth of the signal being sam- 
pled. When the switch is opened, the buffer amplifier's 
output might not have been keeping up with the input 
Signal » but since the voltage at the input of the amplifier 
is now static, the buffer will f.'ven!aally settle out to the 
held capacitor's voltage. 

The problem ivilh building Fig. i] is that it is impossible 
at the present time to build a perfect switch. When the 
switch is opened M is nol truly turned off: it has some 
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residual leakage capacitance and resistance. In hold mode, 
there is some residual coupling to the input signal because 
of this leakage capacitance. This error term is commonly 
called feed thro ugh. Another error term is pedestal voltage. 
The process of turning real -world switches off induces a 
charge transfer that causes the hold capacitor lo experience 
a fixed voltage step |a pedestal) when enleringhold mode. 

Another problem with Fig, 5 is that it is impossible in 
the real world to build a perfect capacitor. Real -wo rid 
capacitors have nonideai behaviors because of dielectric 
absorption and other factors. This dielectric absorption \%ntl 
manifest itself as a pedestal that is different for different 
input-voltage slew rates. Even If Ihe capacitor is refined 
until it is "perfect enough,*' the switch and the buffer 
amplifier may contribute enough capacitance in parallel 
With Chold fhat the resultant capacitance has dielectric ab- 
sorption problems. 

Fig, 6 is an implementatinn of Fig. 3 using real- world 
components. The switch is implemented with a p-channel 
MOS FET. When the drive voltage is - 15V. the circuit is 
in track mode. If the FET has an on resistance of R, then 
the 3-dB bandwidth of the circuit is l/(2irRCh^jjj)' Qi^^ [the 
drain-to-gate capacitance] is always in parallel with Cj^pid* 
so even if Cj^i^jd and the buffer amplifier have low^ dielectric 
absorption, tlie dielectric absorption associated with Cd^ 
\¥ill cause this circuit to exhibit pedestal changes with 
different input signal slew^ rates. 

When the drive voltage is changed to +15V, the FET 
turns off and puts the circuit into hold mode. The drain4o- 
sourcc capacitance (Cj^) contributes feedthrough error 
equal to Qi^/Csj^jij. If the drix^e voltage changes infinitely 
fast, the pedestal error is (30V)[Qg/Cij„|ji), If the drive volt- 
age changes at a slower rate, the pedestal error wull be less, 
but a gain error term will now appear. Assume that the 
drive voltage changes slow'ly relative to the bandwidth of 
the track-and-hold circuit (l/[27TRQ,^jt(i))- Assume also that 
the FET is on until the drive voltage is equal to V,^j and 
that it is off when the drive voltage is greater than Vj,-,. The 
process of going into hold mode begins with the drive 
voltage changing from -15V to +ir5V. As the voltage 
changes from - 15V to V^n, ^ha\d experiences very little 
pedestal error since the current Cti^{dv/dt) mostly flows 
into the FET, which is on. When the drive voltage reaches 
Vjf,. the FET turns off and all of the Q^Idv/dt) current flow^s 
into Cjuji^i, The pedestal in this case is [15V Vj,J{Q|jj^/ 
Chuidl" Notice that this is a smaller pedestal than in the 
previous case where the drive voltage changed infinitely 
fast. Also notice that there is a V,,^ term in the pedestal 
equation. This i.H a gain errtjr. 

Pedestal errors are easy to deal with in the real world. 
There are a number of easy ways to remove offset errors. 



Cii« 



Vir,- 



Gate 

Drive 

~1SV, +15V 



Ij-^ T T — 

Q1 = Cdg ^ ChoM 






Fig. 5. B^sfc tra€k-artd-hQli^ circuit With tdeaf components 



Fig. 6. Basic track-md-hQid circuit with fsBi components. 
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Gain errors are nol necessarily bad either, since ideal gain 
errors can be corrected with a compensating gain stage. 
But because C^jg is a semiconductor capacitance, it tends 
to change value with the applied voltage. This leads to a 
form of error called noniinearity* In general, gain errors 
that are caused by semiconductor capacitances (like that 
calculated in the above paragraph) iead to nonlinearity 
errors, A track-and-hold circuit application that is affected 
by nonlinearity errors is sampling a signal and calculating 
its Fourier transform. Feed through and dielectric absorp- 
tion errors also are hard to deal with- Commonly, a different 
track-and-hold architecture is used to achieve better linear- 
ity, feedlhrtJUgh, and dielectrit; absorption pfirlormance. 

Fign 7 is a diagram of the track-and-hold architecture 
used most often to achieve 16-bit or better resolution along 
with S-MHs', band widths. In track mode the drive voltage 
is - 15V. turning Ql on. The output voltage is the inverse 
of the input voUage. The inverse of the input voltage is 
impressed across Cj,5,|^j during track mode. When the drive 
vol tage is changed to + 1 5 V ^ Ql turns off and V„^j, i s held . 

Fig. 7 has several advantages over Fig- 6. Since the switch 
[Ql] is at a virtual ground point, the pedestal voltage is 
constant with Vi„ and equal to (30V)(Qj^/Ct,y3(j]. This is 
because the drain and source are always at i^ero so that 
when Ql is turned off the same amount of charge is always 
transferred to Ci,,^!^^ Also, since iit! jjoint on Ql moves with 
V^^, the FET doefj not contribute any dielectric absorption 
error terms. 

Fig, 7 does have feedtli rough error. It is equal to ^M^dJ 
C^hoidl' Theoretically this error could be substantially elimi- 
nated if a second switch could be turned on after entering 
hold mode to ground the junction of the two resistors. 
However, a real drawback of this circuit is that the op amp 
Ul has to have the same bandwidth and slew rate capabil- 
ities as the signal being sampled. In the descriptions of 
Figs, 5 and 6 it was mentioned that the buffer amplifier 
need not have the same bandwidth as the signal being 
sampled. So in summary. Fig. 7 eliminates some of the 
errors of the previous circuits but introduces at least one 
new limitation. 

HP 345dA Track-and-Hold Architecture 

Fig, H is a modification of Fig. 6 that has most of the 
advantages aiid very few of the disadvantages of the previ- 
ous circuits. Here the switch is implemented with two 
n-channel JFETs and one p-channel MOS FET, In track 
mode the JFETs Ql and Q2 are on and the MOS FET Q3 
Ls off. Ql and Q2 are on because their gate-to-son rce volt- 
ages are zero, since their gates t:rack V^^^. Their gates track 
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V^„ because in track mode point B is an open circuit and 
CRl and CR2 act like resistances of about 1 kil. CRl and 
CR2 are current regulator diodes, which are simply JFETs 
with their gates wired to their sources. In hold mode, Ql 
and Q2 are off and Q3 is on. Ql is now off because point 
B is now at - 15V and thus the gate of Ql is at - 15V. CR2 
now appears as a current source of high resistance and the 
gate of Q2 is clamped at about 7V below Vqu,, turning off 
Q2. Q3 is on because its gate (point A) is at -15V, 

In hold mode^ feed through error is very low. since the 
feedthrough caused by Q^^^ is shunted into the ac ground 
created by Q3's being on. Also, the pedestal error caused 
by Qi^^ is constant for all Vjj,, since the gate of Q2 is clamped 
at 7V below V^.^i, Since V^^i is tracking V^,, during track 
mode (or will settle out to Vi^ after hold mode is entered), 
the pedestal error caused by Qi^^^ ts ( -^VjlQiga/Choid) ^^d 
has no V^„, dependent terms. Therefore it makes no differ- 
ent;e to the linearity errors of the track-and-hold circuit 
whether C^j^^ is nonlinear with bias voltage. 

It is not so obvious that C.^^^^ contributes almosi nothing 
to the pedestal errors and the nonlinearity ernjrs of the 
circuit, fn addition to being a T-swilch thai reduces feed- 
through errors in hold mode, Ql^ Q2, and Q3 when 
switched in the correct sequence act to remove almost all 
of the pedestal errors caused by C^ji^' This is very important, 
since G^^^^ is nonlinear, and if its pedestal errors remained, 
the linearity of the circuit would be no better than that of 
Fig. 6. Ql is selected such that its pinchoff voltage (Vj^^^^fd 
is greater than that of Q2. Thus* as point B is driven to 
-15V. Q2 turns off before Ql. Once Q2 is off, the only 
coupling path to C^^j^j;! is through the capacitance Ctj^a. 

Fig. 9 shows the various waveforms present In the circuit. 
When Ql is finally turned off, the voltage on CI has a 
pedestal error of (V^,, - 15V)(Crfgt/CJ. This pedestal 
couples into Chnia through Cj^^- The magnitude is (V,,, - 
15V)(Cj^i/Ci)[Cjs2^^hoid) Since Qj^^ is nonlinear and the 
coupling has a V^,^ dependent terin^ the pedestal on Cj-^^i^ 
now has a nonlinear component* But after Ql and Q2 are 
off, point A is driven to -15V, turning Q3 on, C; is now 
connected to V^^^ through the on resistance of Q3 and ap- 
proaches the voltage Vp^jj. This voltage movement, which 
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Fig. 8. HP 34S8A track-and-hold architecture. 
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is of the same magnitude as d's pre%nous change but in 
the opposite direction, couples into Ch^jd through Q^^ and 
totally removes the pedestal error previously coupled into 
Chgid through Cd,2- 

Another point that also might not be so obvious is that 
Qlt Q2. and Q3 do not contribute any dielectric absorption 
errors to the track-and-hold circuit. Since in track mode 
the drains p sources* and galea of Ql and Q2 are at the same 
potential (Vin), none of the PET capacitances has charge 
on it before it is put in hold mode. Therefore, the charge 
transferred to CY,,,[,i through the FET capacitances when 
hold mode is entered is the same for any value or slew rate 
of Vjn, so it doesn't matter whether the FET capacitances 
have high dielectric absorption. 

Summary 

The perl'ormance of the HP 3458 A with sinusoidal and 



nonsinusoidal inputs is known to be very good. The DIVlM 
was tested against a synthesized arbitrary waveform 
generator under development at the U.S. National Bureau 
of Standards which was capable of generating sine waves 
and ANSI-standard distorted sine weaves with an absolute 
uncertainty of 10 ppm. The HP 3458A measured all of the 
various lest waveforms with errors ranging from 5 ppm to 
50 ppm for 7V rms inputs from 100 Hx lo 10 kHz. 

Thfi digital ac measurement capability of the HP 3438A 
combines the best features of the traditional thermal and 
analog computational ac rms techniques in addition to add- 
ing several advantages of its own. Measurement accuracies 
for digital ac are comparable to thermal techniques for botii 
sinusoidal (crest factor 1.4] and large-crest-factor non- 
sinusoidal w^aveforms. Like analog computation, digital ac 
reading rates are reasonably fast compared to thermal rms 
techniques. The major advantages of digital ac include 
linearity superior to traditional analog rms detection 
methods and significantly faster low -frequency rmsac mea- 
surements (less than six seconds for a 1-Hz input). Short- 
term reading stabilitv is excellent, allowing previously dif- 
ficult characterizations to be performed easily. 
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Calibration of an 8y2-Digit Multimeter from 
Only Two External Standards 

Internal transfer standards and autocalibratlon simplify 
external calibration and extend tlie period between external 
calibrations to two years. 

by Wayne C. Goeke, Ronald L, Swerlein, Stephen B. Venzke, and Scott D. Stever 



ONE OF THE EAIU.IEST PRODUCT CONCEPTS for 
the HP 3458A Digital Multimeter was to develop 
a means for calibrating its measurement accuracies 
from only two external reference standard Sh This is not 
possible with the traditional design for a DMM, which 
requires independent adjustment of the full-scale gain and 
zero offset for each measurement range and function. 

Calibration is a process in which individual gain and 
offset values are adjusted, manually or electmnicaUy. to 
yield minimum error relative to an applied input, as shown 
in Fig. 1. Gain and offset calibration values are generally 
determined using precision ratio ti'ansfer measurements 
relative to a smaller set of working standards whose errors 
are directly traceable to national standards. In the United 
States, standards are kept by the National Institute of Stan- 
dards and Technology [NiST], formerly the National Bureau 
of Standards (NBS). Dc voltages are often derived from a 
1.0 18- volt saturated electrochemical cell known as a Wes- 
ton standard cell. The output voltage is divided, or other- 
wise ratioed, to yield other traceable values. For example, 
the output would be divided by 10,18 to produce O.IV. 
The ratio transfer process is, in general, different for each 
calibration value. It is prone to both random and systematic 
errors, which may propagate undetected into instrumenta- 
tion through the calibration process. This calibration (or 
verification] uncertainty will produce a "floor'' measure- 
ment error sometimes equal to or greater than the uncer- 
tainty of the instrument alone. 

The objectives for two-source calibration are to reduce 
this floor uncertainty and to provide an independent 
method to increase confidence in the overall calibration 
process. The HP 3458 A uses a highly linear a nalog-to- digi- 
tal converter (ADC) to measure the ratio between a trace- 
able reference and its divided output. The ADC performs 
the function of the precise ratio transfer device. 

Sources of Error 

The errors in any ratio measurement can be divided into 
two general types: differential errors (D) and integral errors 
(1), A differential error is a constant percent of full scale 
and is independent of the input. These errors are handled 
like dc offsets. An integral error is a function of the inpuh 
and the relationship is usually nonlinear. These errors are 
generally thought of as gain errors. The maximum total 
error can he expressed as: 

Ei(x] ^ I(x/100%) + D, 



where x is the input to the ratio device and Ej(xj is the 
error, both expressed as a percent of full scale. The general 
form of the error bound is shown in Fig, 2. 

What is of concern is the error in the output or measured 
value expressed as a percent of that value. Expressed in 
this form, the maximum error is: 

E^Cx) = I + D(100%/x). 

Where E.^[x] is the total error in the output or measured 
value expressed as a percent of x. The general form of this 
error bound Is shown in Fig. 3. For ratios less than one, 
the tnlcil error is dominated by the differential errors of the 
ratio transfer device. Since the differential error term is 
equal to the differential linearity error multiplied by one 
over the divider ratio, this error growls to infinity as the 
divider ratio ^rows smaller. 

HP 3458 A Uncertainty 

The design goal for the HP 3458 A DMM was for internal 
ratio transfer errors to be equal to or lower than those 
achievable with commercially available external ratio di- 
viders. This set the total ratio measurement error (linearity) 
requirement for the ADC for a 10;1 transfer to approxi- 
mately 0.5 ppm of output or 0.05 ppm of input. 

Fig. 4 illnstrates the integral and differential linearity 
achieved with the HP 3458 A ADC design. The test data 
was generated using a Josephson junction array intrinsic 
voltage standard [see 'Josephson Junction Arrays/' page 
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100% 
Percent of Range 

Fig. 2. Uneaftty srror as a perceni of range 

24). Fig* 4a shows typical deviation from a straight Hne for 
the input voHage range from minus full scale to plus full 
scale expressed in ppra of full scale. This expresses the 
test data in a form dominated by thL^ integral linearity ef- 
fects. Intejiral error less than 0.1 ppm of full scale was 
achieved. Fig. 4b shows typical test data expressed as ppm 
of reading (output). This data indicates differential linearity 
error less than 0.02 ppm of reading. For a lf):l ratio transfer 
the predicted error would be approximately I -h IflD or 0.3 
ppm. Fig* 4c shows measured data, again using a Josephson 
junction array standard to characterize the error at 1/10 of 
full scale relative to a full-scale measured value. The data 
indicates a 10:1 ratio error nf O.ni ppm of the input or 0.1 
ppm of the measured {output) value. This nipresents typical 
results: the specified Soratio transfer error js greater than 
0.3 ppm. Measurement noise contributes additional erroTt 
which can be combined in a root-sum-of-squares manner 
with the linearity errors. 

Offset Errors 

Linear measurement errors in a DMM are of two general 
types, offset errors and gain errors. Offset error sources 
include amplifier offset voltages, leakage current effects 



Differential 
|[>ie0ral 
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Percent of Range 
Fig, 3. LinBarity error as a percent of reading. 
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(IR). and thermocouple effects generated by dissimilar met- 
als used in component construction or interconnection. 
Fig, 5 shows a simplified schematic of the dc measurement 
function. Switches Si and S2 are used to provide a zero 
reference during each measurement cycle. Offset errors 
common to both measurement paths, for example the offset 
voltage introduced by amplifier Al, are sampled and sub- 
Ira cled during each measurement sequence. This is referred 
to as the autozero process. 
GDireclion of the Femalning offset error is achieved by 
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Josephson Junction Arrays 



A Josephson junctton is formed by two superconductors sepa- 
rated by a thin insulating barrier, When cooled to liquid heiium 
temperatures {4.2K), these devices exhibit very complex non- 
linear behavior that has led to a wide range ot applications <n 
anaJog and digitaE electronics, A quantum mechanfcal anatysis 
shows thai these junctions generate an ac current whose fre- 
quency is related to the junction voftage by the relation f = 2eV/h 
where e is the electron charge and h is Planck's constant. When 
the junction is driven by an ac current the effect operates in 
reverse. The junction oscillation phase locks to the applied ac 
current and the junction voltage Socks to a value V = hl/2e. This 
phase locking can also occur between harmonics of the applied 
ac current and the Josephson oscillation. Thus, the junction l-V 
cun/e displays a set of constant-voltage steps [Fig. 1) at the 
voltages V = nhf/2e, where n is an integer The Josephson junc- 
tion thereby provides a means of translating the inherent accu- 
racy of the frequency scale to voltage measurements. 

In July of 1 972 the Josephson effect was adopted as the def- 
inition of the U.S. legal volt For the purpose of thjs definition the 
quantity 2e/h was assigned the vatue 463593.42 GHzA/. Since 
then, tests of the Josephson voJIage-to-frequenGy retation have 
verified its precision and rndependence of experimental condi- 
tions to the level of a few parts in lo'''^.'' 

The Josephson voltage st^idards of 1972 had only one or two 
junctions and could generate voltages only up to about 10 mV. 
This low voltage required the use of a complex voltage divider 
to calibrate the 1 .01 8V standard cells used by most standards 
laboratories. To overcome the limitations ot these low vo ft ages. 



< 



Voltage Step Present 
under RF Excitation 




Voaage ^5V Div) 

Fig . 1 . Partial I - V curve of an 18, 992-junction Josephson junc- 
tion array without RF excitation Also shown is a typfcaf t-V 
curve under 75-GHz excitation, which is a constant-'i/oftBge 
step at a voftage V ^ nhfl2e. The voitage V is between -12V 
and + 12V. and i$ determined by controiiing the bias current 
and source impedance to select the value of n. 

researchers at the U.S. National Institute of Standards and Tech- 
nology (formerly the IMalionaJ Bureau of Standards), and PTB in 
West Germany have developed superconducting integrated cir- 
cuits that combine the voltages of several thousand junctions. 
The most complex of these chips uses 18.992 junctions to gen- 
erate 1 50,000 con stent -voltage steps spanning the range from 
- 12V to + 12V (Fig. 2). The chip uses a f inline lo collect 75-GHz 



providing a copper short across the input terminals. A re- 
ference measurement is taken and the measured offset is 
stored. Values are determined for each measurement func- 
tion and range configuration. The offset is subtracted from 
all subsequent measurements. The HP 3458A performs all 
zero offset corrections by automatically sequencing through 
earh of the required configurations and storing the appro- 
priate offset correction during the external calibration pro- 
cess. These offsets are the b term in the linear equation 
y ^ mx -1- h, where y is the calibrated output result and x 
is the internal uncalibrated measurement. These calibrated 
offsets can be made small and stable through careful printed 
circuit layout and component selection. 

Gain Errors 

Gain errors in a DMM result from changes in amplifier 
gains, divider ratios, or internal reference voltages. Each 
gain term exhibits a temperature coefficient and some finite 
aging rate, and can potentially change value following ex- 
posure to high-humidity environments or severe shock or 
vibration. Periodically, known values close lo the full scale 
of each measurement function and range are applied to the 
DMM lo calibrate the gain ratio m such that y = mx + b 
is precisely equal to the known input value, y. However, 
even after gain calibration* a DMM can easily be exposed 
to conditions that may introduce new errors. The HP 34 58 A 
DlVlM implements a special method for self -adjusting all 
instrument gain errors and many offset errors relative to 
its own Internal references. 



DC Calibration 

Calibration of the dc function begins by establishing 
traceabilily of the internal voltage reference. The internal 
7V Zener reference (see **A High-Stability Voltage Refer- 
ence," page 28) i.s measured relative to an externally applied 
traceable standard. A traceable value for this internal refer- 
ence is stored in secure calibration memory until the next 
external calibration is performed. Next, the gain of the lOV 
range is determined by measuring the internal 7V reference 
on this range. The gain value is stored in secure autocali- 
bration memory. This gain value can be recomputed at any 
time by simply remeasuring the internal 7V reference. The 
stability, temperature coefficient ^ and time drift errors of 
the internal 7V reference are sufficiently small (and speci- 
fied) compared with other gain errors that remeasurement 
orautocalibration of these gains will yield ,smaller measure- 
ment errors in all cases. Adjustment of the full-scale gain 
values of all other ranges relies on the precise ratio measure- 
ment capabilities of the HP 3458A ADC as demonstrated 
in Fig, 4c. For the IV- range gain adjustment, the traceable 
internal 7V reference is divided to produce a nominal IV 
output. The exact value of this nominal IV is measured on 
the previously adjusted lOV measurement range at approx- 
imately 1/10 of full scale. The measured value, a ratio trans- 
fer from the internal 7V reference, is used to adjust the 
gain of the iV range of the dc voltage function. This gain 
value is again stored in secure autocalibration memory. 
Neither the precise value nor the long-term stability of the 
nominal IV internal source is important. The interna! IV 
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Fig, Z The layout for an 18,992- 
juncWn voitage standard array 
cspsble of generatir^g voftage 
steps tn the range of - 12V to 
+ 12V The horizontal lines repre- 
seryt 16 strfphnes, each of which 
jjasses through 1187 juncttorjs. 
The junctions are too small to be 
distinguished on this draw/ng. 



power from a wavegufde and direct it through a set of power 
splitters to 16 striplines, each of which passes through 1187 
junctions. A network of high-pass and low-pass filters allows the 
microwave power to be applied in parallel while the dc voltages 
^d in senes-^ 

In operation, the array is cooled to 4.2K In a liqu^d-hefium 
dewar, A Gunn- diode source at room temperature provides the 
required 40 mW of 75-GH2 power It is possible to select any 
one of the 1 50.000 constant- volt age steps by controlling the bias 
current level and source impedance, A continuous voltage scale 
can be obtained by fine-tuning the frequency. The accuracy of 
the voltage at the array terminals is equal to the accuracy of the 
time standard used to stabilize the Gunn-diode source Actual 
calibrations, however, are limited by noise and thermal voltages 
to an accuracy of a few parts Jn 10^. 

The abihty to generate exactly known voltages between - 12V 
and + 1 2V can eliminate the problems and uncertainties of poten- 



tiometry from many standards laboratory functions. For example, 
Josephson array standards make it possible to perform absolute 
calibration of voltmeters at levels between 0. 1 V and 1 0V without 
the uncertarnty of a resistor ratio transfer from standard cells. 
Another application is the measurement of voltmeter linearity with 
an accuracy higher than ever before possible. 
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source must only be stablr; for the short time required to 
perforni the two measurements of the transfer- 
Each of the reiiiainmg dc voltage ranges is automatically 
gain adjusted relative to the internal 7V reference througfi 
a similar sequence of fuU-scale-to-l/lO-full-scaJe transfer 
measurements. All gain errors can than be readjusted rela- 
tive to the internal reference to remove measurement errors 
at any later time. The only gain error that cannot be adjusted 
during autocall brat ion is the time and temperature drift of 
the internal 7V reference. 

Ohms and DC Current Calibratton 

Calibration of the ohms fii net inns is similar to that of 
the dc voltage function. Traceability for the internal 40-kn 
reference resistor is established first. The internal reference 
resistor is measured relative loan externally applied trace- 
able IQ-kSl standard resistor. The traceable value for this 
internal reference is stored in secure calibration memory 
until the next external calibration Is performed. Resistance 
measurements are made by drivijig a known current 1 
through an unknovi^n resistance R and meas tiring the resul- 
tant voltage V. The unknown resistance value R is com- 
puted from Ohm's law, R ^ V/L Since the dc voltage mea- 



surement function has been previously traceably adjusted, 
only the values of the ohms current sources (1) need be 
determined to establish calibration. 

Adjustment of the ohms current source values begins by 
applying the nominal lOO-microatnpere current source (10- 
kft range) to the traceable 40'kfl internal resistance stan- 
dard. The value of the current source is computed from 
the traceable measurements and stored in secure aiitof:ali- 
bration memory. The 100-/jtA current soun:e can Bk remea- 
sured (autocaiibrated) at any time to correct for changes in 
its value. Residual errors in this autocaiibrated measure- 
ment are reduced to those of the internal reference resistor 
and the aulocalibrateci error of the ItjV dc voltage range — 
essentially the drift uf the internal voltage reference. P'or 
resistatice measurements, only drift in the internal resis- 
tance reference will affect measurement accuracies. The 
gains of the voltage measurements V and the current 
sources h which are derived from the internal vultage refer- 
ence, will also change as this reference drifts, but the com- 
puted value for R is not affected since the V/l ratio remains 
unchanged. 

The known 100-^A current, its value determined in the 
previous step, is next applied to an internal 5.2-kn resistor 
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(an internal 10-to-l ratio transfer measurement). The value 
of this resistor is determin«ci, again from Ohm's law. This 
new resistor R is computed (R = V/1) from the lflQ-/AA 
currenl previously determined relative to known traceable 
standards and the previously calibrated dc voltage func- 
tion. The v^ilue of this resistor is stored in autocalibralion 
memory. This resistor is actually the 10-;iA dc current 
function shunt resistor. With the shunt resistor R traceabi y 
determined, traceable dc current m.easurements can be 
computed from Ohm's law. I = V/R. 

Now that the 5.2-kn internal shunt resistor is known, 
the 1-mA ohms currenl source [l-kfi range) is applied and 
its value computed as a ratio relative to the 100-;uA current 
source value. The 1-mA current source value is stored in 
autocalibration memory. This combined ohms current 
source and dc current shunt resistor ratio transfer process 
continues until all six currents and all eight shunt resistors 
are known relative to the Iwo external standards. 

As we set out to show, all gain errors for dc voltege, 
ohms, and dc current measurements have been traceably 
adjusted relative to only two external standard values: lOV 
dc and 10 kO. Table I summarizes the HP 3 45 8 A errors for 
the internal ratio transfer measurements described so far. 

Table I 
Internal Ratio Transfer Errors 



External 10 Vdc 
Internal 7V 
lOVdc 

External 10 kll 
Internal 40 kH 
Internal 40 kn 
100/iA 



Internal 7V 
lOVdc 
IV dc 
Internal 40 kil 

100 M 

Internal 5.2 kfl 
1mA 



0.03 ppm 
0.02 ppm 
0,33 ppm 
0.30ppm 
0,15 ppm 
0.50 ppm 
0.50 ppm 



Additional Errors 

Gain and offset variations are the dominant sources of 
measurement error in a DMM, but they are by no means 
the only sources of error. Measurement errors are also pro- 
duced by changes in leakage currents in the inpyt signal 
path. These may be dynamic or quasistatic leakages, A 
more complete schematic of the input circuit of the HP 
3458 A is shown in Fig. 6, Recall that switches SI and S2 




are used to null the dc offsets of amplifier Al and its input 
bias current. However, the capacitance CI causes an error 
current l^rr to flow when Si is turned on. This current, 
sourced by the input, generates an exponentially decaying 
error voltage Is^ylR + Rj). If Rj is large, as it is for ohms 
measurements, significant measurement errors can result. 

These errors can be reduced by providing a substitute 
source {shown in the shaded section of Fig. 6) to provide 
the charging current for the parasitic capacitance CI, 
Amplifier A2 follows the input voltage so that when switch 
S3 is turned on between the S2 and Si measurement 
periods- Cl will be precharged to the input voltage. Second- 
order dynamic currents flow because of the gate-to- drain 
and gate-to-source capacitances of the switches, which are 
FETs, The HP 345 8 A performs complementary switching 
to minimize these effects. During an autocalibration, the 
offset of buffer amplifier A 2 is nulled and the gain of the 
complementar>^ switching loop is adjusted to reduce errors 
further 

High ohms measurements are particularly sensitive to 
parasitic leakage currents. For example. 10 ppm of error 
in the measurement of a lO-MI! resistor will result from a 
change of 5 pA in the 500-nA current source used for the 
measurement. Over the 0X-to-55''C operating temperature 
range a 5-pA change can easily occur. During autocalibra- 
tion, which can be performed at any operating temperature^ 
several internal measurements are performed with various 
hardware configurations. The results are used to solve 
simultaneous equations for leakage current sources. Know- 
ing these leakage currents allows precise calculation of the 
ohms current source value for enhanced measurement ac- 
curacy. 

Many other errors are also recomputed during autocali- 
bration. Autocalibration can be performed in its entirety 
or in pieces (dc» ohms, or ac] optimized for particular mea- 
surement functions- The dc voltage autocalibration » for 
example* executes in approximately two minutes. The an- 
tocalibrarion process for the ohms functions, which also 
cahbrates the dc current function, takes about eight minutes 
to complete. If the user is only concerned with correcting 
errors for dc or ac measurements, the ohms autocalibration 



Sll Vin + Vofi 

S2: + Vo3 

Result: (Vtn ^ V^s) - (0 + V^s^ - Vm 




Fig. 5, Sfmpfified schematic of the dc voftage mBRsurement 
function. 



11^* 6. A more complete schematic of the HP 34^A input 
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Fig. 7. ^C Bttenuator gain-versus- frequency character isfic. 

sequence can be omitted to save time. 

AC Frequency Response Calrbration 

The goals for self -calibration of the HP 3458A extended 
beyond the dc measurement functions. Just as the concept 
of sampling a signal and digitally compuling its Irue-rms 
vslue goes against traditional DMM methods, so does the 
idea of adjusting the frequency response and gain of an m: 
voltmeter without applying external ac calihration sources. 
Normally* the first step in the calibration of an ac voltmeter 
would be to adjust the instrument for constant gain at all 
frequencies. This frequency flatness correction is generally 
performed by manually adjusting either resistive or capaci- 
five circuit components. Resistive components usually de- 
termine gains at lower frequencies and capacitive compo- 
nents usually determine gahis at higher frequencies. The 
frequency response f:harocterislic of the HP 34 5^ A ac mea- 
surement function is dominated by five conipensated RC 
divider networks, which are used to condition the input 
signal for e<ich measurement range. The gain-versus-fre- 
qnentiy characteristic of an RC attenuator circuit is shown 
in Fig. 7, Wheri the attenuator is properly compensated 
(T] = T^), the resulting divide ratio is a frequency indepen- 
dent constant determined solely by the resistive elements. 

It can be shown using Fourier transforms that if the input 
to a linear circuit is a perfect voltage step and the output 



of the same circuit is also a perfect voltage step, then the 
circuit transfer function is constant with frequency. The 
hardware used to implement the digital ac measurement 

technique of the HP 3458A is also used to sample a step 
output of the RC attenuator. The sampled data is used to 
compensate the internal RC divider networks for flat gain 
versus frequency without external inputs* 

A simplified schematic for the lOV ac measurement range 
is shown in Fig. B. The active compensation of the divider 
netvvork is achieved by generating a "virtual trimmer* cir- 
cuit element to allow the adjustment of the divider time 
constants. The trimmer is a programmable-gain bootstrap 
amplifier connected across resistor Rl. The variable-gain 
amplifier allows control of the voltage across Rl, effectively 
varying Rl's value. The resistive divuder ratio can be elec- 
tronically servoed to match the fixed capacitive divider 
ratio given a measurable error function. The servo error 
signal is generated by applying an extremely square voltage 
step to the network. The step output is sampled at least 
twice. An amplitude difference between samples indicates 
the presence of an exponential component resuhing from 
miscompensation of the attenuator. The digitally con- 
trolled loop serv^os the difference signal to adjust the virtual 
trimmer to achieve precise cancellation of frequency de- 
pendent errors. Sample times can be optimized for 
maximum sensitivity to the attenuator time constant RC, 
thus improving servo-loop rejection of second -order time 
constants resulting from capacitor dielectric absorption or 
other parasitic effects. 

Sampling of the voltage step uses the same internal tools 
required to perform the digital ac measurement fimction. 
The flatness aulocaltbratit>n voltage step is sampled with 
the integrating ADC configured for 18-bit measurement res- 
olution at 50,000 conversions per second. An internal pre- 
cision sampling time base is used to place samples with 
tOO-ns resolution and less than 100-ps time jitter. Fig. 9 
siu>ws the range of attenuatc^r output waveforms t>re.sent 
during frequency flatness au local ibrat ion. When the at- 
tenuator is compensated correctly, the outpul waveform 
will closely resemble an ideal voltage step as shown. Test 
data has shown that the automated compensation yields 
less than 50 ppni of frequency response error from dc to 
30 kHz. Autocalibration of the frequency response will 
t:orrect for component changes caused by temperature, 
humidity, aging, and other drift mechanisms. Correction 
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A High-Stability Voltage Reference 



Aulocaltbration in the HP 3458A DigitaJ Multtmeter is a process 
of transferrirrg the gain accuracy of a singie voltage reference 
to aii nneasurement gains. The dessgn goai for the internal refer- 
ence of the HP 3458A was to provide iong-term stabiiity and 
temperature stability comparable to external standards that 
would normally be used to calibrate an SV^-digit multimeter. 
These goals were achieved by using a temperatufe-stabilized 
solid-state Zener reference. Without temperature stabiltzaiion, 
the Zener"B voltage drift with temperature is approximately 50 
ppmrC, A proportional temp era tu re control loop senses trie chip 
temperature of the reference device and reduces this drift to less 
than 0.15 ppmrC. 

The long-term drift of each voltage reference assembly is mea- 



flr 



sured by an automated drift monitoring and screening process- 
Reference assemblies, including the temperature controiler, are 
monitored until the aging rate is shown to be less than the 8 
ppm/yr stability requirement of the HP 3458A, Summarized test 
data for a number of B ppm/yr reference assemblies is shown 
in Fig 1 Monitoring the references for additional time allows the 
selection of assemblies that exhibit aging rates less than 4 ppm/yr 
for the high-stability option. 

David E. Smith 

Development Engineer 

Loveiand Instrument Division 
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Fig. 1 . HP 345BA snlemal voltage 
reference drift distributiorj. 



of these errors allows a single specification to apply for 
extended operating conditions. 

AC Gain CaHbration 

Once the fret|iiency flatness characteristics are adjusted, 
the second step of calibration can be completed. Gain cor- 
rection for the measurement must still be achieved. In Fig. 
7 it can be seen that when frequency compensation is 
achieved, the attenuator gain can be established equally 
well at any frequency as long as the calibration signal 
amplitude is precisely known* Adjustment of the circuit 
gain using a dc signal is convenient since a traceably cali- 
brated dc vdltage reference and a dc voltage measurement 
function are available. Gain adjustment of the ac measure- 
ment function using known dc voltages allows complete 
autocall bration of ac measurement accuracy in mucli the 
same manner as the dc voltage measurement function. 

Several mechanisms can limit the accuracy of a dc gain 
adiustment. Dc offsets or turnover errors can be mininiized 
by performing gain adjustment calculations using known 
positive and negative voltages. Errors caused by white noise 



are reduced by averaging 40,000 samples for each voltage 
measurement made through the wide-bandwidth track- 
and-hold circuit. Low-frequency 1/f noise is minimized by 
chopping these 40,000 readings into groups of 1000. each 
group sampling alternating polarities of the known internal 




Fig. 9. The range of atter^uator output waveforms present 
during frequency flatness compensatton. The output 
waveform closely resembles ar} ideal voltage step when com- 
perjsaf/ort is correct. 
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dc calibration voltages. This voltage chop is performed at 
a fast enough rate to achieve maximuni cancellation of the 
If noise voltage. A final error mechanism results from 
aliasing of internal spurious signals. The internal 10-MHz 
clock signal tends to be present in small amounls every- 
where. The ac signal path and the rrack-and-hold circuit 
(2-ns sample aperture) each have sufficient bandwidth to 
couple the internal clock into measurements, if the sample 
spacing Is a multiple of the 100-ns clock period Jhe internal 
spurious clock will be aliased or mixed down to contribute 
a dc offset in the measurement. A 100-/iV-peak spurious 
clock signal can lead directly to a lOO-juV error in measuring 
the internal dc calibration signal as shown in Fig. 10. The 
HP 3458 A uses a random sampling time base mode during 
this calibration sequence. The time base generates ran- 
domly spaced sample Intervais with a resolution of 10 ns. 
The chopped groups of random samples* 40,000 in ail, are 
averaged together to obtain the net gain of the divider. 
Errors caused by dc offsets, w^hite noise. 1/f noise, and 
clock aliasing are reduced using this internal calibration 
algorithm. Gain calibration of the ac measurement function 
relative to the internal dc reference i.s accomplished i^vith 
less than 10 ppm error for intervals extending to two years. 
The residual dc gain calibration error will limit the absolute 
measurement accuracy for low-frequency inputs* 

Additional Errors 

Besides adjusting the frequency response and gain of 
each ac measurement range^ other corrections are per- 
formed during autocalibratioii. Offset voltage corrections 
arc determined for each ac amplifier configuration. The 
offset of the analog true-rms-to-dc converter is determined. 
The offset of the analog trigger level circuit is nulled. Inter- 
nal gain adjustments for vari^ous measurement paths are 
performed. For example, the track-and-hold amplifier gain 
is precisely determined by applying a known dt: voltage 
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and measuring the output in track mode using TVz-digit 
internal dc measurements. A gain ratio is computed using 
this measurement and the hold mode gain is determined 
by averaging 40*000 samples using the 6-;iS, 50.000-read- 
ing-per-second, IS-bit conversion mode of the integrating 
ADC. This gain Is critical So the accuracy of the digitally 
computed rms ac voltage function and to the wideband 
sampling functions. Ac currant measurements use the same 
shunt resistors as the dc currents. A differential amplifier 
is used to sample the voltage across the shunt resistors for 
ac current measurements, and the gain of this amplifier is 
computed during autocall brat ion. 

As a result of autocalibration, the ac measurement accu- 
racy of the HP 3458A is unchanged for temperatures from 
O^'C to 55^C. for humidity to 95% at 40^C. and for a period 
of tu^o years folloviung external calibration. Execution of 
only the ac portion of the autocalibration process is com- 
pleted in approximately one minute* 

Summary 

1 vvo-source calibration of a state-of-the-art digital mul- 
timeter provides several benefits: 
■ Increased process control within the standards iabora- 
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Fig* 10. (a) Using the cfock-dertvsd time base, a 1Q0-{xV 
spunouB clock signal can lead directly to a lOOfiV error in 
measuring ihe internal dc calibration signal (b) The HP 3458 A 
uses a random sampltng time base mode to eliminate tiiis 
error source 



(b) 



Fig .11. (b) Tra ditlonal calibre lion chain for dc and ac voltage 
(b) HP 3458 A calibration chain, showing the Increased verifi- 
cation confider)ce that results from interriat calibration 
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tory through the Independent ralto transfers of the DMM 

■ Reduced calibration time 

■ Increased measurement accuracies in real environments 

■ increased confidence through complete self-testing. 
The greate^sl benefit of two-source calibration is seen not 

by the instrument end user but by the calibration facility 
supporting those instruments. Fig. 11 shows the normal 
instrument and two-source calibrated instrument traceabil- 
Ity chain. When verifying the results of the two-source 
calibration process, the metrologist now has the indepen- 
dent checks of the HP 3458A to catch inadvertent human 
errors in the normal process. Tecbniciue, cabling, and other 
instruments used in the generation of calibration values 
are no longer op en -loop errors that may propagate through 
a calibration laboratory. Two-source calibration can iden- 
tify errors anywhere within the traceability chain, from 
primary standards to final values. 

The HP 34 58 A autocalibration procedures are also per- 
formed during the instrument seif-tesi, which lakes about 
one minute. The only difference is reduced averaging of 
the internal results for faster execution. Also, the results 
are not retained in rnenxory afterward. The self-test proce- 
dures perform highly accurate measurements on each range 
of each function, thereby providing a comprehensive 
analog and digital confidence test of the system. 
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Design for High Throughput in a System 
Digital Multimeter 

High-speed custom gate arrays, microprocessors, and 
supporting hardware and a substantial investment in 
firmware design contributed to the design of the t~iP3458A 
Df^PJi as a system for moving data efficiently. 

by Gary A. Ceely and David J, Rustici 



MANUFACTURERS OF ELECTRONIC and other 
ty^es of products have learned that high test system 
throughput is vital to maintaining production ca- 
pacity. As a primary component of automated test and data 
acquisition systems, the system digital muhimeter (DMIyf) 
has become a major factor in determining system through- 
put. A DMM must not only be able to take and transfer high- 
speed bursts of readings, but must also have the ability to 
reconfigure itself quickly when measuring several different 
parameters in rapid succession. 

Historically, DMJvf performance has been hindered by a 
number of factors, such as relay switching times, ADC con- 
version delays, and the limited processing power of early- 
generation microprocessors. In addition to controlling the 
ADC hardware, taking and transferring readings, and pars- 
ing commands, the microprocessor has been saddled witii 
scanning the front-panel keyboard, updating the display, 
and polling various peripheral ICs to monitor and update 
status information. Increasing demands on the capabilities 



of firmware written for these machines have only com- 
pounded the problem. Adoption of more English-like pro- 
gramming languages has added greatly to both bus over- 
head (because of the length of these commands) and parsing 
time* which formerly was a minor factor. 

Another performance limitation in system DMMs has 
resulted from the need to make floating measurements, that 
is. measurements referenced to the LO terminal instead of 
earth ground. Since the LO terminal may be raised to a 
potential several hundred volts above ground, the ADC 
hardware must also float with this voltage. The problem 
here is that the HP^IB (IEEE 488, lEC 625). and therefore 
the hardware that interfaces to it. is earth -referenced, re- 
quiring that the ADC hardware be isolated from the control- 
ling microprocessor. In many cases, the ADC hardware is 
designed around a second microprocessor w^hich com- 
municates with the main microprocessor via an isolated 
serial link, forminj^ a bottleneck in high speed ADC pro- 
gramming and data transfers. 
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Fig, 1 . HP 345BA Dsgitaf Multimeter system block diegram. 
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Considering the history of DMM perforniance, it becomes 
obvious that the design of the instrument as a system in 
itself is critical to the performance of the surrounding auto- 
matic test system as well. Two key design goals for the HP 
345 8A were that it be able to reconfigure itself and take a 
reading 200 times per second, and that it be able to take 
and transfer readings (or store them internally) at a burst 
rate of 100,000ys. To achieve these goals, system design for 
the HP 3458 A focused on expediting the flow of data 
through the instrument, both in the hardware and in the 
firmware* 

Design Overview 

A yiinfjliiif^tl block diagram of the HP 3458 A is shown 
in Fig. 1. Like previous designs, the DMM is divided into 
two sections, inguard and outguard. which correspond to 
the hardware inside and outside of the guarded (isolated) 
section nf the DMM, In lliis design, however, the bottleneck 
of the serial interface between the two sections is overcome 
by the use of a high -speed (S Mbits/s) fiber optic data link 
and custom gate arrays on each end to decode and buffer 
received data. 

Performance features on the outguard side include an 
8'MHz MC68C0OO main processor, high-speed RAM and 
ROM (requiring no wait states from the processor] , a sepa- 
rate 80C51 microprocessor to control the front -panel Inter- 
face, and a programmable timer used as an operating system 
clock. This represents a significant upgrade in the outguard 
hardware over previous 6800-based designs, and not only 
yields taster execution of instructions, but also frees the 
main processor from polling peripherals, since all I/O and 
interproi lessor communications are now interrupt-driven. 
Additional gains are realized through the use of a double- 
buffered HP-IB input scheme (the parser reads data from 
one buffer while an interrupt service routine fills the other) 
and a hardware HP-ID output buffer* which allows the main 
processor to wtHq data to the HP-IB in words [16 bits) 
instead of b>1:es [8 bits). 

Outguard RAM is divided into three sections: an EEPROM 
for storing calibration constants, standard RAM [nun- 
voiatile), and optional RAM (volatile). Calibration Rj\M is 
distinct from the rest of RAM because it is protected from 
accidental overwrites by a hardware mechanism that also 
makes writing to it rather slow^. Standard RAM is divided 
into program memoryH reading memory (lOK 16-bit read- 
ings), state storage, and system overhead (stacks, buffers, 
etc,)^ Nonvolatile R^\M is used here to protect stored instru- 
ment states, subroutines^ and user key definitions. Optional 
R.^\M is available only as additional reading storage (64K 
readings). 

Inguard hardware is alscj under micmprotiessor control 
(an 80C51, in this case), but the heart of the inguard section 
is a 6000-gate, 20'MH2 CMOS gate array. Functions per- 
formed by the gate array include communications with the 
outguard section through a custom UART, trigger logic con- 
trol, analog-to-digital conversion, and communications be- 
tween the UART and other parts of the inguard section. 
Shift registers are incorporated to minimize the number of 
interconnections between the gate array and other inguard 
circuits (the ADCt the ac and dc front ends, and the trigger 
control logicj. Five shift registers containing 460 bits of 



information reduce the number of interface lines to just 
three per circuit- Communjcalions are directed by the pro- 
cessor, which also interprets messages sent from the out- 
guard section and generates response messages (see "Cus- 
tom UART Design," page 36. 

Firmware Structure 

The division of tasks between the inguard and outguard 
processors is based on the need to minimize the flow of 
messages between them. Inguard firmivare is responsible 
for controlling the ADC measurement sequence, controlling 
the trigger logic during measurements, and directing con- 
figuration data to the other inguard circuits. Outguard 
firm^vare responsibilities are as shown in Fig. 2. Primary 
functions^ such as parsing, command execution, display 
updating, and keyboard input are performed by separate 
tasks under operating system control. Other functions, such 
as HP-IB 1/0 and interprocessor communications, are inter- 
rupt-driven, are coded in assembly language for maximum 
speed, and communicate with the primary tasks via signals 
and message exchanges. High firmw^are throughput is 
achieved by focusing on optimization of time- intensive 
tasks, such as data transfer and manipulation^ parsing and 
execution of commands, task switching overhead, and the 
measurements themselves. 

Fig. 3 shows the flow of data through the HP 3458A. 
Data flow is divided into two main paths: the input path 
for messages received from the controller, and I he output 
path for measurements generated by [he instrument. When 
a controller sends a command such as DCV 10, the data flow 
is from the controller to the HP 3458A through the HP^IB. 
The HP-IB handler accepts incoming data and passes it on 
to the outguard processor's parser^ which interprets the 
command and then passes control to an execution routine. 
After determining the necessary actions, the execution 
routine sends slate change data to RAM and inguard -bound 
messages to the UART. Messages sent to the inguard .section 
are of two types: measurement messages, which control 
the type of measurement [e-g., dc voltage or ac voltage], 
and configuration messages, which define the state of the 
front ends and the ADC and tinier control circuits. Data is 
received by the inguard UART and passed to the inguard 
processor, which parses the message and either acts upon 
it or directs it through the communication controller to 
one of the other inguard circuits. Once the r;onfiguration 
phase is complete, the ADC is ready to take a reading* and 
throughput becomes a matter of getting the reading out of 
the in.strujiient quickly. Referring again to Fig, 3, the output 
data path is from the ADC to the inguard UART, through 
the fiber optic link, and on to the outguard processor. The 
processor performs any required math and formatting up- 
erationSj and then directs the data either to reading storage 
or to the HP-IB. 

Data input, Configuration, and Measurements 

programming commands coming in over the HPdB are 
received and buffered by an interrupt ser\dce routine, 
which in turn signals the HP -IB parser/ execution task. The 
interrupt code is designed to continue reading characters 
from the HP-IB chip as long as they £:ontinue to come in 
at a rate of 100 //.s.ciiaracter or fasten In this manner, an 
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Firmware Development System 



Firmware tor the HP 3458A DMM was developed on four HP 
9000 Computefs (Models 320 and 350) under ttie HP 84000'UX 
mcfoprocessor development environment Each system was 
fully equipped to opefaie as an independent devetopment sta- 
tion, and the systems were networked to facjfitate transfer of 
code revJsioPcS (see Ffq i ) A frfth station was used tor consolidat- 
ing code modiflc^ions to be tested using a prototype HP 3458 A 
and the HP 3458 A production test system After passing an ex- 



IEEE 602.3 LAM 



lensive battery of tests, code was released in EPROM torm for 
other prototype instruments 

Firmware tasks were divided along lines intended to minimize 
interdependence between ttie designers Tne areas of responsi- 
bility were (1) n^asuremems and calibration, (2) digitizing. (3) 
data processing, formatting, and storage, and (4) parsing. I/O. 
and operating system overhead Rg 2 shows a breakdown of 
Ihe amounr of object code generated by various modules Ai- 
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entire command or string of commands can be read in 
during a single invocation of the interrupt routine, thereby 
gRnerating only one signal to the parser task. In reality r two 
input buffers are used: one that is filled by the Interrupt 
routine, and another that is read by the parser task. After 
the Interrupt routine signals the parser that data is present 
in one buffer* that buffer belongs to the parser task, and 
the other buffer is used for the next command that Lomes 
in. Wben the parser empties a buffer, that buffer is freed 
for later use by the interrupt routine. Using two buffers 
simplifies pointer manipulation so that data can be read 
in and passed to the parser quickly- 

To maximize the flow of data to the HP-IB parser/execu- 
tion task^ the instrument musl first be programmed to an 
idle state [e.g., using TARM HOLD). This allows the operating 
system to keep the HP-IB parser task active so that no task 
.■i witching is necessary when an HP- IB command is re- 
ceived. The parser is a table-driven SLR (simple left-right) 
design, with all critical components coded in assembly 
language. Simple commands can be parsed in as little as 



1 ms; longer commands take as much as 3 ms. For a further 
increase in system throughputp command sequences can 
be stored as subprograms, in which case they are first com- 
piled into nssenibiy language by the parser/code generator. 
Executing command sequences in this fashion eliminates 
most of the overhead of bus I/O and parsing and allows 
the HP 3458A to perform reconfiguration and trigger oper- 
ations almost twice as fast as the same sequence with indi- 
vidual commands (340/s instead of 180/s). 

In many situations, the HP 345 8 A will be reconfigured 
for a different measurement setup with each test, which 
may include only one measurement. The setup changes in 
these cases may take more time than the measurement f so 
the configuration time must be minimized. To perform 180 
reconfiguration and trigger operations per second, the in- 
strument must be able to transfer, parse, and execute a 
command in slightly over 5 ms. Of this total, several 
hundred microseconds are spent in bus transfer and system 
overhead, and up to 3 ms may be spent parsing the com- 
mand- Given that an additional several hundred microsec- 
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Fig. 2, Outguard firmware modules. 

together, over 28,000 lines of C code were written, representing 

roughly 80% of the 356K bytes of object code generated. Ttie 
renrrainder (12,000 lines) was written in 68000 assembly lan- 
guage. 

During the most Intense period of firmware development, code 
revisions were released on a weekiy basis To relieve the firmware 
team of the time-consuming tasi^ of generatEng and testfng code 
revisions, a fifth team member was given this responsibility 
Fimnware designers uploaded source code weekly to the fifth 
system, where ft was compiied. linked, and downloaded lo an 
emulator. Having source code avaiiabie on this system made it 
possible to trace and analyze defects using a dedicated QA 
system to reproduce them. The fifth development system was 
also used for archiving firmware revisions using RCS (UMIX revi- 
sion control system). To reduce duplication of effort, the test 
system used tor firmware development was a repiica of the HP 



3458 A production test system, which had been developed earlier 
in the project cycle to be used in environmentaJ testing and 
prototype characterization. 

As the firmware construction phase neared completion, two 
engineers were added to the project so that test software could 
be developed in parallel with the firmware effort. To save test 
writers the trouble of learning the deiaiis of test system operation, 
drivers and utilities were written that allowed each new test to 
be written as an isolated subroutine. The test system executive 
simply loaded and ran each test as it was needed, thereby pro- 
viding an effscjeni mechanism for adding new tests throughout 
the construction and lest phases Both hardware and firmware 
designers wrote tests for the test suite Each was assigned a 
specific area of functionality to be tested, using both while-box 
and black-box approaches, 

in addition to the tests written spec if tea I ty to verify firmware 
operation, each revision of code was subjected to the production 
test software {which mainly tested the analog hardware for mea- 
surement accuracy), Additional test coverage included the entire 
HP 3458A user's manual, with emphasis on the command refer- 
ence, example programs, and randomly generated combinatfons 
of valid and invalid synta!< As defects were found, they were 
fixed and the test code run again tor verification. Following a 
successful run through the test suite, code was released and 
source code was saved using RCS. Saving old code revisions 
enabled the firmware team to recreate earlier code revisions to 
help track down defects that may not have been reproducible 
on a newer code revision When a new detect was found, tests 
were written and added to the test suite to ensure that the defect 
would not recur. By the end of the project, the test suite had 
grown to where 1 2 hours were required to run atl tests. To assess 
testing progress and effectiveness, defects were submitted to 
HP's DTS (defect tracking system). Methc reports were gener- 
ated and analyzed on a weekly basis to help assess the firmware 
status. 

Victoria K. Sweetser 

Development Engineer 
Love I and Instrument Division 



onds will be spent taking and tratisf erring the reading, only 
aboiit 1 ms is left for the execution of the command. In 
this millisecond, the execution routine must range-check 
parameters, calculate the gain and offset values, and config- 
ure the trigger controller, the ADC. the front-end hardware, 
and the inguard processor. In the w^orst case, performing 
these operations taki^s considerably longer than a mil- 
lisecond. A complete configuration cif all tht^ inguard sec- 
tions takes 1 .4 ms, and settling time for the front-end relays 
adds another 1 .3 ms. In addition, a function command may 
require as many as ,six floating-point calcuIalH>n,s, each 
taking G.3 ms. This all adds op to well over 4 ms; therefore, 
a number of optimizations have been incorporated to re- 
duce configuration time. 

The first step is to avoid reconfiguring the instrument or 
a section of inguard if there is no change. For example, if 
the present hniction is at: voHs and the new command is 
ACV. only the range is configured (if it changes], not the 
function. The ADC configuratjon is the same for dc voltSt 
ohms, and dc current, so the ADC section is tiot reconfi- 
gured for changes between these functions. The trigger con- 
figuration changes only for digital ac voltage or frequency 
measurements, so a new configuration is sent only when 



entering or leaving these functions. Tn general, reconfigtira- 
tion occurs only to the extent required by a given command. 

Each combination of function and range uses different 
gain and offset values for the ADC readings. The gain and 
offset values are scaled by the ADC's aperturep so If the 
aperture increases by 2. the gain and offset are scaled by 
2. An execution routine retrieves the gain and offset values 
from calibration memory and scales them by the aperture. 
Then the 120%- and 10"/o-of- full-scale points are calculated 
for overload detection and autoranging. The autoranging 
algorithm uses a different ADC aperture and has a separate 
set of 120% and 10% points. These two calculations were 
removed from the execution routine, and are done at cah- 
bration time since the autoranging algorithm alwm-^s uses 
the same ADC aperture. To reduce the effect of the other 
four calculations, a data structure is used that saves the 
gain and offset for each function and range as it is needed. 
If the aperture of the ADC is changed, the data structure 
is cleared* and as function and ranges are revisited, the 
data .structure is filled in. This eliminates recalculation of 
values that are constant for a given aperture. 

An operation that is not always necessary hut takes con- 
siderable time during a range or function change is a special 
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sequence of relay closures in the Iron I -end circuitn\ This 
sequence protects the relays from damage when high volt- 
age is on the input terminals during range changes, but is 
oot needed when measuring low voltages or if high voltage 
is only present when the instrument is set to a high voltage 
range. Therefore, the HP 345flA provides an HP-IB program- 
mable command to defeat the protection scheme, speeding 
up the relay sequence by a factor of five. If an oven'oltage 
condition occurs while protection is inactive, an interrupt 
is generated and the relay sequence is reversed, thereby 
protecting the relays from damage. A delay of 0,4 second 
is then inserted to prevent a rapid recurrence of the over- 
load condition, and the instrument reverts to normal (pro- 
tective) relay sequencing thereafter. 

Another technique used to reduce the configuration time 
is to defer computations until the last possible moment. 
The scale factor used in the format conversion of ADC 
readings from integer format to real or ASCII format is an 
example of this technique* Many commands cause the scale 
factor to change, so instead of each command computing 
the scale factor, a flag is set and the calculation is performed 
when the scale factor is first used. This eliminates wasted 
time from unnecessar^"^ calculations when many inter- 
mediate configuration changes are sent to the instrument, 
and reduces the time spent responding to even a single 
HP-fB command. 

Data Oow between the outguard and inguard sections 



has the potential to be a bottleneck, because the UART and 
the inguard processor can only accept configuration data 
at a rate of 20,000 u^ords/s. Furthermore, commands to 
change relays can take a millisecond for the inguard proces- 
sor to execute. To relieve the outguard processor of the 
need to wait on the inguard processor, a buffer was added 
to store messages bound for the UART. This buffer is d^p 
enough to hold an entire configuration change — 128 com- 
mands. This allows the outguard processor to overlap its 
activities with the inguard processor's. If the buffer is empty 
and the UART is not busy sending data, the 68000 will 
send a command directly to the UART. avoiding the over* 
head of the buffer. If the UART is busy, data is written to 
the buffer instead. In this case, the UART generates an 
interrupt when it is ready to accept the next word, which 
is then retrieved from the buffer and sent. 

In addition to fast reconfiguration, system throughput 
depends on the time required to make a measurement. Fig, 
4 shows the steps an ADC reading goes through before it 
is sent to the HP- IB. The first step is autoranging: if a reading 
is less than 10% of the range or greater than 120%, the 
instrument switches to the next range, changes the ADC's 
aperture for a fast measurement, and takes a reading. This 
procedure is repeated until the correct range is found, and 
then the final measurement is made with the ADC's original 
aperture. Although this algorithm is very fast (typically 8 
niilliseconds). it usually requires that the ADC take several 
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Custom UART Design 



At the center ot the corrimunscatpons link between the inguard 
and outguard sections of the HP 3458A DMM is the custom 
UART (universal asynchronous receiver/transmitter) A seriaj in- 
terface was chosen because an isolated parallel interface would 
have been prohibitively expensive. Unfortunately, conventional 
UARTs are too slow to meet the HP 3458 As required data rate 
of 200 kbytes/s, which corresponds to a baud rate of 2 Mbits/s, 
counting start and stop bits. Therefore, fiber optic couplers were 
chosen, which also provide the benefit of infinite isolation resis- 
tance 

Conventional UARTs require a clock rate that is 16 times the 
baud rate; thus, to generate the 3458A's required baud rate, the 
clock rate would have to be 32 MHz. The 16>< cEock rate is 
needed to compensate for mts matched clock frequencies and 
waveform distortion. These two factors can be controlled within 
the HP 3458 A, so a clock rate of three times the baud rate is 
used The UART design is implemented as pan ofaCtvlOSgate 
array dhven by a 10-MHz clock. This clock rate yields a baud 
rate of 3.3 Mbits/s, which meets the design goal with some mar- 
gin 

The data format tor the UART is shown in Fig. 1. The first bit 
is the start bit and indicates the beginning of a message The 
next bit is the handshake bit. If this bit is high, a data/command 
message will follow immediately If the bit is low, the message 
Is a handshake and the next bit will be a stop bit. A handshake 
message is sent each time a data message is read by the pro- 
cessor, ensuring that a new message will not be sent until the 
previous message has been read. The next- lo- last bit is the inter- 
rupt or command bit, used to indicate whether the preceding 
message was data or a command. A command message from 
the inguard section could be an ADC conversion failure, an end 
of sequence message, or a change in the front or rear terminals 
Command messages generate interrupts, eliminating the need 
for software to check the data from the UART to determine the 
message type The middle 16 btts of the message represent the 
data or command, and the last bit is the stop bit. 

Fig. 2 shows a block diagram of the UART and the communi- 
cation controller When the decoding state machine detects that 
a start bit has been received, it waits three cycles to decide 
whether the message is a handshake. If so. the state machine 
returns to its initial state. If the message is data, the next 16 bits 
are clocked into the tnpui shift register. The state machine then 
examines the next bit (the command/data bit) If the message m 
a command, an interrupt is generated. 
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Fig, 1, interprocessor msssaQG formats. 

For transmitted messages, the encode machine first generates 
a start bit If the message is a handshake, the next bit is set high. 

otherwise {if the message is dataj, the next bit is set low. The 
16 bits of data are sent next (if required), and if the message is 
a command, the last bit is set high 

Buffers in the UART are used both for received data and data 
to be transmitted This aitows the ADC to leave data in the buffer 
while starting the next measurement, thus maximizing the ovehap 
between oatguard and mguard Once the buffer has been 
emptied, the handshake message is sent and an interrupt can 
be generated The interrupt can be used as a request tor more 
data to be sent. The buffer queues requests from four sources. 
the ADC s error detection circuitry, the ADC's output register. 
the trigger controller messages, and the inguard processor. 

The input buffer also has a direct output mode to the shift 
registers. When data is sent to the mguard section, the processor 
is Interrupted, the data is parsed, and, if the message is a con- 
tig u ration message, the direct output mode is selected in the 
communication controller. This mode allows the next message 
to be sent to both the processor and the shift register, thereby 
sending the configuration data directly to the appropnate section. 
In this case, the processor receives the message but does not 
act upon it, thereby eliminating the overhead of processor inter- 
vention rn the configuration process 

Aithough the use of microprocessors has enabled instruments 
to offer greatly enhanced measurement capabitity, a severe 
speed penalty may be incurred if firmware is burdened with tasks 
that are best Jeft to hardware. The HP 3458A's use of a custom 
UART coupled directly to the measurement hardware optimizes 
performance by balancing the workload between hardware and 
firmware, 

Davfd J. Rusttci 

Dave lop men I Engineer 

Loveland Instrument Division 




#-^ ADC Controller 
ADC Offsei 
Tngger Controller 
DC Front End 
AC Front End 



80C51 
Processor 



ADC Data 

ADC Error 

'#^^ Trigger Control 



Fig. 2. Bfock diagram of the 
UART and data communication 
pofttons of the foguard gale array 



3d HEWLETT-PACKARD JOURNAL APRIL 1989 



)Copr. 1949-1998 Hewlett-Packard Co. 



samples to generate one reading. Therefore, a faster mea- 
surement will be made if autoranging is tmned off. 

Throughput is also enhanced by minimizing operating 
system overhead. In cases where high throughput is not 
an issue (e.g., long integration times), measurements are 
handled by a backgmimd task, which runs whenever the 
instrument is not actively executing commands. This task 
simply monitors the trigger and trigger arm states to see if 
a measurement should be taken. When throughput is an 
issue, however, measurements are initiated directly by the 
HP-IB command parser/execution task. In this case, the 
overhead of task switching (approximately 250 pus] is elimi- 
nated, leaving only the overhead of communication be- 
tween the interrupt service routine and the HP-IB task. 
Another speed enhancement is the use of preprogrammed 
states, which fail into two categories: predefined states (ac- 
tivated using the PRESET command j, and user-defined 
states (stored using the SSTATE command and activated 
using the RSTATE command), Since these commands cause 
an extensive reconfiguration, their primary benefit is in 
putting the instrument in a known desired state. However, 
they can also save time when the alternative is to send 
long strings of commands to program the instrument to the 
same state. 

Output Data Path 

Once the instrument has been configured and triggered. 
a measurement is Idken by the ADC and transmitted 
through the fiber optic link to the outguard processor. The 
format for this reading is either a 16-bit or a 32-bit two's 
complement result with the range offset subtracted. The 
next step is to convert the readings into volts, ohms, or 
amperes by multiplying by the gain of the range, if a math 



Comp uter 'Coni roller 



operation is active, it is initiated using a procedure variable 
that points to the math subroutine. Al this point, the reading 
is in a 64-bit floating-point format, and a format conversion 
is required for an integer. ASCII, or short real format. The 
last step is to display the result and send It to memory* or 
the HP-1£. Some steps can be eliminated using the appro- 
priate HP-IB command; for example, the display operation 
is deleted using the DISR OFF command. 

If autoranging, math, and the display are turned off and 
the output format matches the ADC's internal format, the 
measurement can he sent directly to the HP*IB or memor>^ 
Special assem.bly language routines were written to handle 
these high-speed modes. The time allowed to read the mea- 
surement and send it out is 10 fxs [given a maximum reading 
rate of 100,000 per second]. There are two data paths: one 
that sends readings to memory* and one that sends them to 
the HP-IB. 

Reading Storage, The memory structure dictated by HP's 
multimeter language is a general circular buffer in which 
readings may be added or removed at any time. This buffer 
can be used In either of two modes: FIFO (first in, first out) 
or LJFO [last in, first out), the main distinction being that 
the LIFO mode w^ill overwrite the oldest readings when 
memory fills, whereas the FIFO mode w^ill terminate w^hen 
memory fills » thus preserving the oldest samples. A general 
program loop for receiving readings from the ADC and 
writing them into memory is as follow^s: 

■ Walt until the ADC has taken a reading. 

■ Write the reading into the current fill location and incre- 
ment the fill pointer. 

■ Has the fill pointer reached the top of memory (buffer 
pointer wrap-around}? 

■ If memory is full and the memory mode is FIFO, stop. 

■ Terminate the loop when the end of sequence is sent. 
Within 10 fxs, the 68000 will allow only about three 

decisions to be made. Even using hand-optimized assembly 
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Fig. 3. input and output data flow paths. 
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language, a single program loop could not be written to 
implement the general memory model in the allotted time. 
The solution uses the fact that if enough decisions are made 
before the start of the bursty the number of on4he-fiy deci- 
sions Cdu be reduced. Before the start of a burst of samples, 
it is known how many readings can be added before the 
buffer pointers wrap around ^ and how much room is left 
before the circular buffer fills. The problem is divided into 
a set of special cases. For example, assume that IDOO read- 
ings are expected from ihe ADC. Memory fill and empty 
pointers indicate space for 2000 readings, but the fill 
pointer is only 100 samples from buffer wraparound. Under 
these conditions, the memory fill algfjriihm can be stated 
as follows: 

■ Fill memory with samples until the buffer fill pointer 
reaches the top of memory. 

■ Wrap around the fill pointer to the bottom of memory, 

■ Fill memory with samples until the sequence is com- 
plete. 

■ Exit the routine. 

Any memory scenario can be expressed as a combinatipn 
of the following special-case loops: 

■ Fill memory with samples until the fill pointer reaches 
the top of memory, then wrap around the fill pointer to 
the bottom of memory. 

■ Fill memory with samples until memory is full (fill 
pointer = empty pointer). 

■ Fill mem^ory with samples until the sequence is com- 
plete. 

Four factors influence the algorithm used: memory mode, 
number of readings expected, total available memory, and 
number of samples before wraparound. All possible com- 
binations of these factors can be accommodated using only 
len special-case combinations. Any particular special case 
can be built out of one to four of the routines listed above. 
Routines are linked together by pushing their addresses 
onto the stack in the rever.se of the order in which they are 
to he executed (tlie address of the exit routine is pushed 
first], and the first routine is called. In the example above, 
the first routine is called to fill memory until it detects 
buffer wraparound. It then loads the fili pointer with the 
address of the bottom of memory and executes an RTS (re- 
turn from subroutine) instruction, which pops the address 
of the next routine from the .stack and jumps to it. The next 
routine continues filling memory until the burst is com- 
plete, then terminates in another RTS instruction, which 
pops the address of the exit routine. The exit routine per- 
forms some minor cleanup (restoring pointers, setting flags, 
etc. J and leaves. 



HP-IB Output, The high-speed output routine for the HP-IB 
uses some of the same concepts as the memory routines. 
In this case, the algorithm is as follows: 

■ Initialize pointers. 

■ Wait until the ADC has taken a reading, then enter the 
readings, 

■ Wait until the HP-IB buffer is ready to accept more data, 

■ Transfer the reading to the HP- IB buffer. 

■ Terminate the loop when the end-of -sequence command 
is sent. 

The HP -IB buffer accepts a l&-bit word from the processor 
and sends the lower eight bits to the HP-IB interface chip. 
Once thi.s byte has been transmitted, the HP-IB chip signals 
the buffer, and the buffer then sends the upper eight bits 
without intervention from the processor. Use of a buffer 
relieves a congestion point in the output data flow that 
would occur if the processor wrote directly to the HP-IB 
chip, since the HP-IB is an eight-bit bus while all other 
internal data paths are 16 bits wide. Using this scheme, 
the HP 3458A is able to offer complete memory and HP-IB 
functionality at the full speed of 100,000 IB-bit dc voltage 
readings per second. 

Summary 

Achieving high throughput in a system DMM is a matter 
of designing the instrument as a system for moving data 
efficiently. Hardware and firmware must be designed as 
integral elements of this system, not as isolated entities. In 
the design of the HP 345 8 At experience with DMM perfor- 
mance limitations provided invaluable insight into key 
areas of concern. As a result, significant improvements in 
throughput were achieved through tlie development of 
high-speed custom gate arrays for ADC control and inter- 
processor communications. Use of high-performance mi- 
croprocessors and supporting hardware also contributed 
greatly to meeting design goals, as did the substantial in- 
vestment in firmware design and development that was 
necessary to translate increased hardware performance into 
increased system performance, 
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High-Resolution Digitizing Techniques 
with an Integrating Digital Multimeter 

Capabilities and limitations of the HP 3458A Digital 
Multimeter as a high-resolution digitizer are summarized. 
Performance data is presented for selected applications. 

by David A. Czenkysch 



WITH ITS INTEGRATING anaJog-to-digital con- 
verter (ADC) capable of making 100,000 conver* 
sions per second, the HP 3458A Digital Multi- 
meter (DMM) raises the possibility that, for the first time, 
a voltmeter can satisfy many requirements for high-resolu- 
tion digitizing. 

What are the chaxacteri sties of a high -resolution digi- 
Mzer? Digitizing requires a combination of fast, accurate 
sampling and precise timing. It also needs a flexible trigger- 
ing capability. The HP 3458 A allows sampling through two 
different signal paths, each optimized for particular appli- 
cations. 

Converting a signal using the dc volts function (vt^hich 
does not use a sampJe-and-hold circuit, but depends on 
the short integration time of the ADC) provides the highest 
resolution and noise rejection. The direct sampling and 
subsampling functions, which use a fast-sampling track- 
and'hold circuit, provide higher signal bandwidth and 
more precise timing. 

High-Resolution Digitizer Requtrements 

As the block diagram in Fig. 1 illustrates, a digitizer 
consists of an analog input signal conditioner followed by 
a sampling circuit. A trigger circuit and tinie base generator 
controls the timing of samples. The output of the sampling 
circuit is converted to a number by an analog-to-digital 
converter (ADC). Once converted to a number, the sample 
data can be processed digitally and displayed to the user. 

Many types of instruments fit this definition of a digi- 
tizer, including digital oscilloscopes, dynamic signal ana- 
lyzers, and digital multimeters (DMMs]. Digitizing products 
can be roughly differentiated by four characteristics: analog 
signal bandwidth, sample rate, signal-to-noise ratio (which 
can be expressed as effective bits of resolution), and type 
of data displayed [time, frequency* etc.]. In general, digital 
oscilloscopes lend to have high bandwidth and sample rate 
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and relatively low resolution, v^hile DMMs and dynamic 
signal analyzers tend to have much higher resolution and 
correspondingly lower bandwidth and sample rate. 

Digital oscilloscopes are known for their high bandwidth, 
typically 100 MHz or greater, and their digitizing rates of 
50 megasamples to one gigasample per second, making 
them useful for capturing ver\^ fast, single-shot events. 
Their resolution of five to eight effective bits is well-suited 
for displaying waveforms on a CRT, since one part in 200 
is perfectly adequate for the human eye. 

Dynamic signal analyzers, on the other hand, are used 
in applications that call for higher resolution — typically 
10 to 14 bits. Examples include dynamic digital-to-analog 
converter testing, telecommunications. SONAR, and seis- 
mic or mechanical measurements that require digital signal 
prot;essing. These applications require higher resolution 
and typically involve frequency- domain analysis. There- 
fore, to judge the attributes of a high -resolution digitizer, 
we should also examine the characteristics of discrete 
Fourier transforms (DFTs) performed on the digitizer's out- 
put data. 

Digitizer Spectral Attributes 

"Effective bits" is a measure of the resolution of an ADC. 
Essentially p it is a measure of the signal-to-noise ratio in a 
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Fig. 1 . GenBraified block diagram of a digitizer. 
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digitizing system expressed as a power of two. This can be 
expressed mathematically as: 

N(effective) = {S/(N + D) - 1.8)/6,02 

where S/(N + D) is the ratio of the signal power of a full- 
scale input to the total power of noise plus distortion, ex- 
pressed in dB. Notice that the effective bits rating and the 
signal-to-noise ratio expressed in 6B are both logarithmic 
scales re la led by the constant fi.02. This ineans that increas- 
ing the resolution of a measuremeni by one effective bil 
results in a 6-dB improvement in the signal-to-noise ratio. 
The system noise term^ N + D. is the rms result of the 
power contributions of harmonic distortion and noise from 
various sources. For an otherwise noise-free^ distortion 
free-system, there is minimum noise component because 
of the fundamental quantization error of the ADC, If this 
IS the only source of error, the number of effective [jits 
approaches the basic resolution of the ADC Fig, 2 shows 
how the number of effective bits decreases as errors from 
other sources increase. 

Other types of errors will appear as random noise. These 
include noise in the input signal, noise in the analog input 
circuits, random jitter in the timing of samples, and noise 
and differential nonlinearity in the ADC. 

Linearity error Is a measure of the deviation of the output 
of an ADC from the ideal straight-line relationship it should 
have with the input voltage. Fig. 3 shows a graph of the 
linearity error of a typical ADC as a function of input volt- 
age* Integral linearity error is the large-scale bow in the 
total linearity error plot. This deviation from a straight line 
can often be described by a second-order or third -order 
function. Differentia] linearity error^ on the other hand, has 
no large-scale structure, so it looks very much like noise. 

If the noise in a digitizer is truly random, then a point-by- 
point average of many independent ensembles of waveform 
data taken with the same input signal will reduce this noise 
by the square root of the number of ensembles, provided 
the different ensembles of data have the same phase re- 
lationship to the input signal. Analog noise in the input 
amplifier and ADC and noise caused by random timing 
errors tend to be uncorrelated with the input signal, and 
so can be reduced by waveform averaging. On the other 
hand, differential linearity error in the ADC and systematic 
timing errors, while appearing like random noise in a single 
pass of data, are repea table from pass to pass, and so are 
correlated with the input and cannot be reduced by averag- 
ing. This provides a w^ay of determining if the signal-to- 
noise ratio of a given digitizing system is dominated by 
input noise or by differential linearity error. 

Effective Bits from the DFT 

OoH way lo characlerize the signal-to-noise ratio of a 
digitizer is to sample a quiet [low-noise] and spectrally 
pure full-scale sine wave and perform a discrete Fourier 
transform {DFT] on the resulting data. The dynamic range 
(in dB) from the peak of the fundamental to the noise floor 
of the DFT gives an idea of the lou^-level signals that can 
be resolved. The level of the noise floor depends on the 
number of frequency points (bins) in the DFT, and hence 
on the number of samples taken, since if the same noise 



power is spread over more frequency bins, there wdll be 
less noise power per bin. 

The DFT spectrum can be used to produce an estimate 
of the signal-to-noise ratio of a digitizer by performing es- 
sentially the same measurement digitally that a distortion 
analyzer performs electronically. A distortioa analyzer 
supplies a inw-distortion sine wave as the input to a circuit 
under test. A notch filter is used to remove the fundamental 
frequency from the output signah The power in the filtered 
signal is measured and a ratio is formed with the total 
output power of the circuit under test. A distortion analyzer 
measurement assumes that the power in the filtered output 
signal is dominated by harmonic terms generated by distor- 
tion in the circuit under test. In practice, however, the 
analyzer is unable to separate this power from the power 
contribution of wideband noise, and hence is actually- 
measuring the signal-to-noise ratio of the output signaL 

An analogous operation can be performed on the DFT 
spectrum of a digitized pure sine wave. A certain number 
of frequency bins on either side of the fundamental peak 
are removed from the DFT data. The data in each of the 
other frequency bins is squared (to yield a power term] and 
summed with similar results from the other frequency bins 
to calculate the total noise powder. The data within the 
narrow iMud around the fundamental is squared and summed 
to give the total signal pow^r. The ratio of these two terms. 
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Fig. 3. Lsneanty errors m an ADC. (a) tritsgra! linearity error. 
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expressed in dB, can be used to compute the number of 
effective bits of resolution of the digitizer. 

Calculations of effective bits from DFT spectra wilJ show 
variations if the test is performed repeatedly. This variation 
can be reduced if the spectral values from many Indepen- 
dent trials are averaged point by point {as opposed to av- 
eraging the time-domain data). Spectral averaging will not 
reduce the level of the noise floor in the DFT data, but only 
the amount it varies. Therefore, if enough ensembles of 
spectral data are averaged, the number of effective bits 
caiculated w^ll] converge to a single number. 

Fig. 4 shows the DFT for 4096 samples of a mathemati- 
call\^ generated ideal sine wave quantized to 16 bits 
(±32.767 counts). From this^ we see that a perfect 16-bit 
digitizer ivill show a noise floor of about -127 dB when 
quantization error is the only source of noise, if the signal- 
to-noise ratio is calculated using the method described 
above, the result is 97.0 dB, or 16.0 effective bits, which 
is ^vhat we would expect. 

Other types of digitizer errors can show up on a DFT 
plot. Distortion reveals itself as harmonic components at 
multiples of the fundamental input frequency. This can be 
distortion in the input signal, hfirmooic distortion in the 
input amplifier, or integral nonlinearity in the ADC. As 
mentioned before, integral linearity error can be approxi- 
mated by a second -order or third-order term in the transfer 
fund ion of the ADC. These higher-order terms generate 
spurious harmonic components in the DFT spectrum. 

Other spurious signals can show up in the DFT spectrum 
besides harmonic distortion. Internal clock .-signals can pro- 
duce unwanted signal components (spurs) either by direct 
cross talk or ihrnugh intermodulation with the input signal. 
These effects are commonly grouped together into a single 
specification of spurious DFT signals. 

Effect of Sample Aperture 

Another aspect fjf dtgi fixers that shuuld be considered 
is the effect of the finite acquisition time of the sampling 
circuit that provides the input to the ADC- This is typically 
some type of sampln-and-hold or track-and-hold circuit. 
For maximal lime certaijily. an ideal track-and-hold circuit 
would acquire a voltage instantaneously when triggered to 
take a sample. In reality, of course- all sampling circuits 
require some finite time to acquire a sample. This sampling 
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function can be approximated by a rectangular window of 
time T over which the input signal is sampled. 

The Fourier transform of a square pulse defined over the 
inten^al — T 2 ^ t ^ T'2 in the time domain has the form 
[sm(iTfT)j -iTfr, which is the familiar function sioc|fT). This 
means that sampling a signal for a time T is equivalent In 
the frequency domain to multiplying the input spectrum 
by the fimclion sinclfT). Fig. 5 shows that the spectral 
envelope of the sine function approximates a single-pole 
low^-pass filter with a 3-dB corner frequency oii^ ^ 0.45 T- 

From this analysis we can conclude that making the sam- 
ple time as short as possible produces the flattest possible 
response because \i maximizes the aperture roll -off corner 
frequency. A less desirable trade-off, however, is that this 
also increases the equivalent white noise bandwidth of the 
sampler, thereby increasing its sensitivity to noise. There- 
fore, in applications where noise is a greater problem than 
frequency roll -off, it would be desirable to have a wider 
sample aperture to reduce the noise bandwidth- 

The transform above %vas defined for a square pulse ex- 
tending from -T/2 to T/2. Since a real sampler cannot 
anticipate its input, the sample must actually occur over 
the interval ^ t ^ T. This implies that any sampler that 
acquires a signal over a nonzero time interval T wnll intro- 
duce an apparent time delay equal to T/2 to the output. In 
most real applications* however^ this distinction is not sig- 
nificant. 

Another characteristic of the sine function that can be 
useful is that its transfer function goes to zero at all frequen- 
cies that are multiples of i/T. This means the sampler will 
reject all harmonics of a signal whose fundamental period 
is equal to the sample aperture. Therefore, a selectable 
aperture allows the rejec^tion of specific interference fre- 
quencies that may be present in the measurement environ- 
ment. 

HP 3458A Digitizing Characteristics 

Many of the same design characteristics required to make 
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Rg, 4. Dfscrete Fourier transform of an tdeaJ sme wave sam- 
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quer^cy resulting from sampling wth an aperture of width T^ 
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Time Interpolation 



To implement the subsampting (effective time sampling) re- 
quired for the hP 3458 A DMM's digitai ac measurement 
technique, some means of synchronizaiion with the input signal 
was necessary. To minimize errors caused by aliasing of ihe 
sampled data, a tJme base with 10-ns resolution was desired. 
However, the intemal 1 0-MHz clock would only allow a sample 
resoiution ot 100 ns relative to a synchronizing trigger event. 
These design requirements dictated the development of the time 
interpolation circuit of the HP 3458 A. 

The insirument's lO-MHz clock is used to generate sample 
liming pulses of variable period in 100-ns (10-MH^) steps. The 
lime interpoJator extends the resolution of the time base from 
100-ns steps to 10-ns steps for initial burst delays {the delay 
from a trigger event to the stah of sampling). This enables the 
HP 3458A to digitize signals with spectral content up to 50 MHz 
without introducing aliasing errors. 

The time interpoJator, Fig. 1 . uses analog techniques to convert 
lime to stored charge on a capacitor. Before an input trigger. 
Ihe JnterpoJalor is reset by shorting both capacitors {S1 and S2 
dosed) with the current source shorted to ground (S3 and S4 in 
position B). An asynchronous input trigger, generated either by 
the ac path's trigger level circuit or by an external ingger input, 
initiates charge accumulation on CI by opening SI and setting 
S3 and S4 to position A. This charge accumulatfon process con- 
tinues until the next positive edge of the 10-MHz clock occurs. 

On this edge 33 and S4 switch to position B. forcing the ac- 
cumulated charge to be held on C1. This charge, Qi, is directly 



proportional to the elapsed time (T^^n) between the mput trigger 

and the next 10-MHz clock edge. Likewise, the voitage across 
C1 (Vvari) 'S ^Iso propoftional to Tvati, which varies between 50 
ns and 150 ns depending on ihe timing of the asynchronous 
input trigger relative to the interna) 10-MHz clock. 

The interpolator remains in this '"hold" stale for an integral 
number of clock cycles, T^^\^y. The next positive-going clock 
edge after Tdeiay initiates the second charge accumulation pro- 
cess. At this lime, S2 opens and S3 and S4 are switched to 
position A. Dunng this time, the same charge^ Q^, is accumulated 
on C1 and C2. This process continues until the voltage on CI, 
Vramp. crosses the programmable comparator threshold Vj. This 
transition generates an output trigger that signals Ihe track-and- 
hold circuit in the ac section to enter hotd mode, thus acquiring 
a sample for subsequent ADC conversion. By programming Vj 
to various values, the system can after this delay jn increments 
of 10 ns, allowing precise timing of a burst of samples relative 
to an asynchronous starting event. 

The output trigger also switches S3 and S4 to position B. This 
not only turns off the current source, but also creates a loop 
between C2, R1, and the buffer amplifier's jnput and output. 
Feedback forces a current through C2, removing its accumulated 
charge, Q2 The resulting current flows through both Cl and 02, 
removing the charge Q^ from capacitor 01 . The process com- 
pletes with CI holding the original charge, Qi. which is proper- 
tional to the delay between the first trigger and the rising edge 
of the internal 10-MHz clock. During Ihe ADO conversion, (a 
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Fig. 1 . HP 3458 A DMM time inter- 
poiaior block and timing dia- 
grams. 
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mininnum of 20 fxs for subsampling) the rime base Cfrcuit waits 
an interval lunmj before repealing ilie charge/disc liarge cycle. 
The accuracy of the 1 0-ns increments is ensured by calibration 
of the circuit gain. Since the lime interpoiator's absoiute delay 
is a function o! I^^^p- Cl , and V,. many variabies can prevent the 
1 0-rrs fncrements Trom being exactly one tenth of a 1 00-ns time 
base step. Interpoiation for ten 10-ns intervals must precisely 
equal one 100-ns clock penod [10 MH^) to minjmize sampling 
errors. By adjusting l^gp-jp (Fig. 2), the slew rate and threshold 
errors are adjusted to yield lO-ns steps within ±50 ps. Time jitter 
fs hefd to less than 1 00 ps rms. Low temperature coefficients for 
C1 and the DAC that generates V, ensure interpolator accuracy 
over the operating temperature range The Urr\e interpolator ts 
adjusted by applying a 2-MHz sine wave to the input and execut- 
ing a calibration routine which atternately programs iOO-ns de- 
lays into either the time base or the time interpolator. By adjusting 



the DAC that controls l.^mp. the routine converges the two delays. 
This tjme base performance contributes to the a noise floor 00 
dB beJow the tundamantal and spurious signals below ^ 55 dB. 

The design of the lime interpolator circuit was reiined using 
analog simulation methods on an HP 9000 Model 320 Computer. 
Computer-aided engineering provided timely feedback during 
developmenL allowing rapid evaluation of alternative circuit to- 
potogtes. Critical design characterizations, difticuit to achieve by 
tfaditional means, were performed accurately and simply using 
CAE simulations The resulting circuit performance exceeded 
our original design goals. 

David E Smith 

Development Engineer 

Loveland Instrument Division 



high-accuracy ac and dc measurements also allov\r the HP 
3456A to perform well as a high-resolution digiti/.er. For 
instancen because it makes true-rms ac measurements using 
digital tet:hniques, it has a scope- 1 ike trigger level circuit 
for waveform synchronit^fition. A precise trigger timing cir- 
t:uit allows sample intervals to be specified lu a resolution 
of 100 nanos«f:onds anti initial delays fnmi a trigger event 
to the first sampli> of a burst can be specified to a resulutioji 



of 10 nanoseconds using an analog time interpolator. 

7\s Ihe hlo{:k diagram in Fig. 6 shows, the HP 3458 A 
provides two dislinct input paths for digitizing, corres- 
ponding to the two amplifiers used for the dc volts and ac 
volts functions- Each path has advantages find disadvan- 
tages. The dc input path should he used when maximum 
resolution and noise rejection are required and the 
bajui width of the input signal is rela lively low. Bet ;a use it 
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uses a track-and-hold circuit, the ac input path can be used 
on signals of higher bandwidth or when the signal must 
be sampled at a very precise point in lime. 

High-Resolution DC Input Path 

The dc input path allows higher-resolution sampling as 
well as a higher single-shot measurement Kpeed, providing 
16-bit samples at up to 100^000 samples per se<:und. The 
bandwidth of this amplifier varies from 50 kHz. to 15U kHz, 
depending on the range selected. The widest bandwidth 
is available on the lOV range, when the amplifier is operat- 
ing at unity gain. In this path, the sampling function is 
performed by the ADC itself with its seiectable integration 
time [sample aperturej. Historical! y, digital multimeters 
with integrating ADCs have allowed only a few discrete 
values for integration time. These values were chosen to 
be multiples of the pnwer-line frequency — the most com- 
mon signal to interfere with a. high-resolution voltage mea- 
surement. In the HP 345 8 A, integration times can be 
specified from 500 ns to 1 s in increments of 100 ns. This 
allows the rejection of an interference signal of arbitrary 
frequency that may be present in the input, and provides 
attenuation of other frequencies above the sample rate by 
the approximate single-pole roU-off characteristic of the 
sample aperture's sine function. The longer the integration 
aperture specified, the more resolution is provided by the 
ADC. Fig. 7 shows the resolution that can be obtained for 
a given aperture. 

Because the dc input path is designed for extremely low 
noise, low offset, and part-per-million (ppm) accuracy, the 
DFT spectra produced in this mode are quite good. In factt 
it is difficult to determine whether the harmonic distortion 
and noise floor measurements are dominated by the HP 
345 8 A or by the inpot signak 

Fig. 8a shows the DPI' calculated on 4096 samples of a 
1-kHz waveform aquired at a rate of 50,000 samples/s with 
an integration time of 10 microseconds. The noise floor 
and spurious DFT signals are below -120 dB, and har- 
monic spurs are below - 106 dB. If the signal-to-noise ratio 
is computed from the spectral data, the result is approxi- 
mately 93.9 dB, yielding 15.3 effective bits. 

The input signal for this test was provided by the oscil- 
lator output of an HP 339 A Distortion Measurement Set, 
whose distortion at this frequency is specified to be less 



Input 
signal^ 



Precision 

DC 
Afnplitier 



Wideband 
AC 

Amplifier 







Track-and- 
Hold CIrcuU 



im^rating 

Analog-to-Oigilal 

Converter 



^Bits 



; t 



Threshold 

Delectton 

Circuit 



Analog 

Time 

liiierpotator 



Triflger 



Trigger Conlrot 

and Sampling 

Time Base 



External Trigger - 



than -9$ dB at the first harmonic. It is unclear whether 
the first-harmonic term at - 107 dB is caused by distortion 
in the source signal or distortion in the HP 3458 A at this 
.sample rate. Elowever, tests performed at slower sample 
rates (and greater resolution) also exhibit this second-har- 
monic term. 

The averaging effect of the relatively wide sample aper- 
ture (10 ^s vs, 2 ns) reduces random noise contributions 
to the DFT noise floor to a level comparable to those of 
systematic oon linearities. Because of this, waveform av- 
eraging only provides an extra 4,4 dB improvemeot in the 
signai-to-ooise ratio, yielding an extra 0.7 effective bit. Fig, 
8b shows the DFT spectrum that results if 64 waveforms 
are averaged. 

A striking example of the high-resolution digitizing capa- 
bility of the dc volts sampling mode involves measuring 
an ultralow-distortion signal source used to characterize 
the performance of seismic measurement systems. The out- 
put of the source is a 0.03 -Hz sine wave whose noise and 
harmonic distortion products are guaranteed by design to 
be at least 140 dB below the level of the fuiuiamentaL 
Superimposed on this is a 1 -Hz sine wave whose amplitude 
is 120 dB below the level of the 0.03-Hz signal. The goal 
of the measurement system two-tone test is to be able to 
see the 1-Hz tone clearly in the presence of the full-scale 
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[WV peak] input without introducing extraneous distortion 
prodycts. The HP 3458A was used to acquire 4096 samples 
with ail ADC aperture of 20 milUsecofids and a sample 
iBterv'^ai of 50 inillisecoods. resulting in a resolution of 24 
bits [7y2 digits}. 

The DFT plot in Fig, 9 shows the result of this test. Only 
a portion of the full 10-Hz bandwidth is shown to make 
the component at 0.03 Hz more apparent. The 1-Hz spike 
at - 120 dB is cleajly visible above a iiobe floor of - 150 
dB. If the 1-Hz component is notched out along with the 
0.03-Hz fundamental, and the remaining power is consid- 
ered noise, a signal-to-noise calculation yields 19.6 effec- 
tive bits. As before, it is not clear %vhether the DFT noise 
floor in this measurement is dominated by noise in the 
input signal or noise in the HP 3458 A. If the rms noise of 
the HP 345 8 A is characterized with the same ADC aperture 
[20 ms) and a quiet dc source is substituted as input , mea- 
surements demonstrate a performance of 22 effective bits. 
The HP 3458 A is clearly capable of verifying the perfor- 
mance of this source to the levels guaranteed by its design- 
ers. We are lold that earlier measuremenls had never been 
able to acliie%^e these low levels of noise and distortion. 

In the dc volts mode, the input signal is sampled directly 
by the ADC. The sampling is synchronous with the instru- 
ment's internal 10-MHz clock. Thi,s leads to a 1 00- na no- 
second peak uncertainty in the time latency of a sample 
or group of samples relative to an external or level 1 rigger 
event. While a time uncertainty of tOO nanoseconds from 
an asynchronous trigger event is perfectly adequate for 
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transform (DFT) of a l-kHz mput stgnai (b) DFT for 64 aver- 
aged acquisitions of fhe hkHz input signal, Effective bits are 
15 3 for (a) and 16.0 for (b) 



most applications, other applications require more precise 
sample timing. 

Digital AC Input Path 

The ac input path provides a wider analog bandwidth 
and more precise timing than the dc path, The bandwidth 
of the ac amplifier is 12 MHz on all ranges except the 
10-mV and 1000 V ranges, where the bandwidth is 2 MHz, 
Autocalibration gxiarantees a frequency response flatter 
than 0.01% [0.001 dB) throughout the frequency band from 
200 Hz to 20 kHz, making this path ideal for characterizing 
frequency response in the audio band. While the maximum 
single-shot sample rate of 50,000 samples per second is 
somewhat lower than the dc input path because of the 
additional settling time required by the track-and-hold cir- 
cuit, a precise timing circuit allows effective time sampling 
(subsampling) of repetitive input signals with effective 
sample inter\^ais as short as 10 ns. 

Achieving true-rms ac measurements with 100-ppm ac- 
curacy using digital techniques requires an extremely 
linear track-and-ho!d circuit. This sametrark-and-hold cir- 
cuit provides 16-bit linearity in digitizing applications. A 
sample acquisition time of approximately 2 ns results in a 
3-dB aperture roil -off frequency of at least 225 MHz, This 
means that amplitude errors caused by the sample aperture 
are insignificant through the entire measurement band. The 
timing of the track-aod-hold circuit is controlled by an 
analog ramp interpolator circuit which operates asynchro- 
nously with the inter[ial 10-MHz clock, giving a burst-to- 
bursl timing repeatability error less than 100 picoseconds. 
The time interpolator allows programming of delays from 
an external or internal trigger with a resolution of 10 ns, 
allowing single samples to be timed very precisely. 

While the greater equivalent noise bandwidth of the 
input amplifier and track-and-hoid circuit results in fewer 
effective bits of resolution in a single-shot measurement 
than Ihu dc input path, the DFT performance for this path 
is still quite good. Fig. 10a shows a typical 2048-point DFT 
plot for a 1-kH'^ sine wave sampled al the single-shnl limit 
of 50^000 samples per second. A sij5nLil-lo-noi?>e ratio calcu- 
lalion on this data yields 10.4 effect ivn bits. The ac input 
path has a greater equivalent noise bandwidth than the dc 
input path, so random noise dominates the signal-to-noise 
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Measurement of Capacitor Dissipation Factor Using Digitizing 



No capacitor outside of a textbook exhibits the iheoretiGal cur- 
rent- to-voftage phase lag ol 90 degrees. This is another way of 
saying that in the real world all capacitors are lossy to some 
extent. These losses are caused by a number o( factors, such 
as fead resistance and dielectric hysteresis 

At a given frequency, the dissipation factor of a capacitor [S 
defined to be the ratio of the equivalent series resjsiance (ESR) 
and the capacitive reactance. Dissipation factor is important for 
many applications. At high power levefs. capacitors with poor 
dissipation factor can overheat. The precision of capacitively 
compensated attenuators can be compromised by dissipation 
factor. Also, the capabilities ol traok-and-hold circuits are de- 
graded by the dissipation factors of their hold capacitors. 

There are two common ways to measure dissipation factor In 
the first method. Ihe impedance of the capacitor under test (CUT) 
is measured at a given frequency and the deviation in phase 
angle from the ideal 90 degrees is used to calculate the dissipa- 
tion factor Bridges are another method used to measure dissipa- 
tion factor. In essence, the CUT is in a bfidge with three other 
capacitors, one of which is adjustable in both C and ESR. When 
the bridge is nuOed. the values of the adjustable C and its ESR 
determine the dissipation factor of the CUT. 

The ac attenuator in the HP 3458A uses a 20'pF capacitor that 
has a dissipation factor requirement of 0.0001 (0.01 %) at 10 kHz. 
Connmerciany available automated equipment exhibits reading- 
to-reading noise of 0.01% and dissipation factor accuracies of 
0.04%. This is inadequate to screen this capacitor reliably. High- 
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Fig. 1. Measuring phase shfft in a stne wave. 

quality manuai bridges can do this job, but their operation is not 
well-suited to a production environment. 

By making use ol the high-resolutfon digitizing and precision 
ac measurement capabilities of the HP 3458 A, it is possible to 
construct an automated dissipation factor meter that is capable 
of making accurate and stable 0.001 % dissipation factor mea- 
surements and capacitance measurements thai are stable to 
001 pF. 

Circuit Description 

In Fig. 1. a method of determining the phase shift of a sine 
wave relative to an external timing pulse occurring at the sine 
wave's zero crossing is shown. Theoretically, onfy V, is needed 
to determine this phase shift The advantage of using a second 
sample (V^) spaced one half cycle later in time is that (V^ - V^) 



measiirement to a much greater extent. Because of this, the 
noise floor can be lowered another 20,6 dB by waveform 
averaging, producing 13.8 effective bits as shown in Fig. 
10b. 

The ac input path supports two digitizing functions: di- 
rect sampHng and subsampling, which is also referred to 
as effective time sampling. The article on page 15 describes 
the subsampling technique. Subsampling allows the sam- 
pling of repetitive w^aveforms with effective sample inter- 
vals as .short as 10 ns, thus allowing the user to take full 
advantage of the 12-MHz analog input bandwidth. The sub- 
sampling parameters are somewhat complex to calculate 
for an arbitrary effective interval and number of samplesj 
but the user need not understand the details of the al- 
gorithm. All that need be specified is the desired effective 
sample inter\^al and number of samples, and the HP 3458 A 
will compute the number of passes, the number of samples 
per pass, the delay increment per pass, and the ADC sample 
rate required to complete the task most efficiently. Further- 
more, if the samples are directed to the instrument's inter- 
na] memory- they will be sorted into the correct time order 
on the fly. 

If the number of samples required for a subsampled mea- 
surement exceeds the size of the instrument's internal 
memory, the samples can be sent directly from the ADC 
to a computer via the HP-IB. Since the HP 3458A cannot 
sort the data in this mode, the samples recieved by the 
computer generally will not be in the correct time order. 
If this is the case, the waveform can be reconstructed in 



the computer's memory using an algorithm requiring three 
sorting parameters supplied by the HP 3458 A. 

Subsampling is essentially the same as direct sampling 
w^hen the effective sample rate is less than or equal to 
50,000 samples per second. Why, then, is direct sampling 
even offered? The answ^er is that the subsampling technique 
only allows sampling biased on the internal time base, 
whereas the direct sampling function includes all the same 
trigger modes as the dc volts function. This means that the 
user can supply an external time base via the external trig- 
ger input to allow sampling at odd frequencies that cannot 
be realized with the 100-iis quantization of the internal 
time base. An example would be the 44.1 -kHz sample rate 
required by many digital audio applications. Direct sam- 
pling is also useful for acquiring single samples with 
minimum time uncertainty^ Samples can be precisely 
placed with 10-ns delay resolution relative to an external 
trigger event and with 2-ns rms lime jitter. \Measurement 
of Capacitor Dissipation Factor Using Digitizing" on this 
page show^s an example of these measurement capabilities 
of the HP 3456A. 

HP 3458A Limitations 

Since the HP 345 8 A must be a voltmeter first and a digi- 
tizer second, it Is not surprising that it has some limitations 
as a digitizer. Perhaps the most significant is the lack of 
an anti-aliasing filter. Because no single filter could be 
included to cover all possible sample rates, and because it 
would degrade the analog performance, the design team 
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Fig, 2* Orcuit to measure dissipation factor. 

is insensHive to voltage offsets on the sine wave. 

Fig 2 shows a circuit using the tecl^^ique of Fig. 1. A sine 
wave IS applied to one of two capacitive divfders. one formed 
by the CUT and Ciow arid the otlner formed by C^^gh ^^^ ^\ow i^ 
provides {he dc bias tor the buffer amplifier). This sine wave js 
aJso applied to a comparator that detects zero crossings. The 
output of the comparator is routed to the externa ( trigger input 
of the HP 3458A and the output of the buffer amplifier is applied 
ro the input of the HP 3458A. The HP 3458A can use its ac 
section to measure the rms value of this output waveform and 
thus Vp m Rg 1 can be determined very precisely The HP 3458A 
can also measure the period of the output waveform and set up 
sample timing parameters to sample the output sfne wave relative 
to the external trigger signal as shown in Fig. 1 . Thus all the 
information is present to determine the phase shift of the sine 
wave through the capacitor divider network. 

The absolute ptiase shift of one side of the capacitor divider 
is not the information desired, however What is desired is the 
phase shift caused by the dissipation factor of the GUT in the 
divider formed by the CUT and 0,^^, This will provide the informa- 
tion needed to determine the dissipation factor ot the CUT 
Computing the difference between the absolute phase shift of 



Drm reference divider (Ct-gn af>d C;,^*) and the mput divider (CUT 
fflxl C^ is the first step towards determinmg the phase shift in 
the input divider resuittng from the dissipation factor of the CtJT 
The HP 3458A 3 EXT otJT Output is used to select either the 
reference drv^Oef or the input dfvider Taking the phase difference 
between it^e reference and input measurements removes errors 
caused by ttne buffer amplifier and the comparator If C>^^ had 
zero dissipation factor, CUT had the same capacitance value as 
Cri»gti* and the switching relay was perfect, this phase difference 
would be entire^ a result of the dissipation factor of the CiJT if 
this phase difference is <^, the dissipation factor of the CUT is 



DF ^ tan(<^) 



(CUT + C^) 



In general, the CUT will not be the same size as C^,^^, C^fegn 
will not have zero dissipation factor, and the switching relay will 
not be perfect However, these conditions are easily controlled 
The feedth rough capacitance of the relay in Fig 2 can iDe reduced 
by implementing the relay as a T-switch If the CUT is different 
in magnitude from Ch^gh^ ^ phase difference will be measured 
even if the CUT has zero dissipation factor. Thks ts because the 
phase shift of the parallel combination of R and Chigh and 0,av, 
is different from that of the combination of R and the CUT and 
Cfoy^. This error can be removed by appropriate correction factors 
implemented in software Also, in general, the dissipation factor 
of the CUT Will not be zero A zero calibration against a reference 
capacitor can remove this error 

Summary 

The precision digitizing capabilities of the HP 3458 A DMIVI 
have been applied to make a traditionally difficult measurement 
of capacitor dissipation factor. Test results show measurement 
accuracies approachtng 0.001%. This corresponds to a phase 
error of 0,0005 degree or a trme error of 150 ps at 10 kHz Also, 
since the capacitance of the CUT is computed as part of the 
dissipation factor calculation, accurate capacitance measure- 
menlB are also generated that are stable to 0.001 pF. 



Ronaid L Swarietn 

Development Engineer 

Loveland Instrument Division 



decided it would be impractical in include one. 

Another limitation is the latency from an external or 
internal trigger to the start of sampling. The ramp time of 
the analog lime interpolator produces a minimum delay of 
at least 400 ns. This means that If the input frequency is 
greater than about 500 kHx, the signal edge that is used to 
synchronize the waveform in a subsampled measurement 
will not even show up in the output data. Oscilloscopes 
typically include some form of analog delay to match the 
timing of the signal path to the trigger circuit, but again 
this was not compatible with the requirements of a high- 
precision DMM. 

Another effect inherent in the design of the analog time 
interpolator is voltage droop. Essentially, the phase of the 
input signal relative to the internal 10-MHz clock is rep- 
resented by a voltage stored on a hold capacitor, which is 
captured at the beginning of a measurement burst and held 
thrniighout Iht* burst. Since there will always be some leak- 
age in the circuits attached to this node, the voltage on this 



capacitor will slowly leak off, causing an apparent length- 
ening in the time between samples. This produces an appar- 
ent frequency modulation in the output data* which con- 
tinues until the charge leaks off completely, at which time 
l;he sample interval will again be stable. This droop rate 
gets worse as leakage increases with higher temperature. 
Measurements on a typical unit at room temperature show 
a droop rate of about 500 ns/s, which persists for about 140 
ms. In other words, during the first 140 uis of a reading 
burst, a sample interval of 20 ^s will be lengthened by 
about 10 ps per sample* 

Waveform Analysis Software 

CJne factor limiting the effectiveness of the HP 34 58 A as 
a stand-alone digilixer is the lack of a built-in CRT for 
waveform display. This shortGoming has been addressed 
with a software library that turns an HP 3458A and a com- 
puter into a real-time single-channel digital oscilloscope 
and DFT analyzer 
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The optional waveform analysis librajy allows a user 
with an HP 9000 Series 200 or 300 workstation or an IBM 
PC/AT-c:onipatible computer with HP BASIC Language Pro- 
cessor to display waveforms in real time. In addition, 
routines are included to perform parametric analysis, 
waveform comparisons, and FFT spectral calculaHons and 
to store and recall wa^^^eforms from mass storage. 

The real-time osciiloscope subprogram, ScopeSen began 
as a means to demonstrate how quickly waveforms could 
be acquired by the HP 3458 A and displayed. It soon became 
an indispensable tool in the development of the ADC and 
high-speed firmware. Since the program had proven so 
valuable during development, we decided it should be in- 
cluded in the waveform analysis library. A user interface 
was added to give the look and feel of a digital oscilloscopet 
inckiding horizontal and vertical ranging, voltage and time 
markers, and an FFT display mode. The program can ac- 
quire and plot waveforms at a rate of approximately 10 
updates per second — fast enough to provide a real-time feel. 

The heart of the Scope58 siib program is a set of 
specialized compiled subroutines for fast plotling, averag- 
ing, and interpolation of waveforms. Since speed was the 
overriding design consideration for these routines, most of 
these subnmtines were written in MC68000 assembly lan- 
guage rather than a higher-level language like Pascal or 
BASIC, The fast plotting routine, in particular* required 
certain design compromises to achieve its high speed. It 
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Fig. 10, (a) Typtcaf DFT for 2048 sampfes of a J kHz sine 
wave sampled at the HP 3458 A ac path's smgie-shot hmtt of 
50.000 sampies per second. Effective brts are 10.4 (b) Effec- 
tive bits can be increased to 13.8 by avemging data for sev- 
erai acquisivons 



uses a simplified plotting algorithm which requires that 
there be one sample per horizontal display pixeh which 
means that the only way to change the horizontal scale is 
to change the sample rate unless the waveform data is 
interpolated to Increase its time resolution before plotting. 
Also, the plotting routine bypasses the machine independent 
graphics routines and writes directly to the bit-mapped 
frame buffer of the graphics screen, This makes the routine 
fast, but it complicates tke programming task* since a spe- 
cial version of the routine must be written for ever>* sup- 
ported display interfacre. 

In addition to the Scopes 8 subprogram, the waveform 
analysis library includes routines that help with waveform 
acquisition, analysis, and storage. Since the HP 345 8A is 
capable of synch rnni^ing with external switching instru- 
ments like a normal DMM, it can be switched to at:qnire a 
waveform per channel in a multichannel data acquisition 
system. This feature, combined with the waveform analysis 
library, can be used to make many complex measurements 
in automated test applications. 

The ]ibrary*s analysis capabilities include routines to 
extract parametric data such as rise time, pulse width* over- 
shoot, and peak-to-peak voltage, and routines to compare 
waveforms against high and low limit arrays. There is also 
a compiled utility for calculating Fourier and inverse 
Fourier transforms. This routine can compute a 204e-tinie- 
point-to-1024'frequency-point transform in as little as 1.2 s 
if the computer's CPU includes a 68881 floating-point co- 
processor. Finally, routines are provided for the interpola- 
tion of vt'aveforms using Ihe time ccmvnlution property of 
the sinclxj function. This technique is common in digital 
oscilloscopes, and allows the accurate reconstruction of 
waveforms with frequency components approaching the 
Nyquist limit of half the sampling frequency. 

The precision digitizing characteristics of the HP 3458A 
and the display capabilities of the waveform analysis li- 
brary combine to form a powerful waveform analysis tool 
in R&D or automated te.st applications. For instance, an HP 
3 45 8 A together with a digital pattern generator can be used 
to test digital-to-analog converters (DACs!. The waveform 
comparison capability of the waveform analysis librarv can 
be used to provide a pass/fail indication. Assuming a DAC 
settling time of 10 /.iS and an HP 3458A measurement time 
of 20 ^ts (only 10 ^ls of which is spent integrating the input 
signal], all codes of a 14-bit DAC fU>,3S4 levels] can be 
acquired in approximately 328 ms. ' The dynamic charac- 
teristics of the 11 AC can be tested using the FFT library 
routine. The DAC can be programmed to output a sine 
wave, which the HP 3458 A can digitize. A DFT on the 
resulting data can be analyzed to characterisce the DAC ft>r 
noise floor and total bannonic distortion [THD]. 

Summary 

The capabilities of a high-resolution digitizer can best 
be characterized by examining its performance in the fre- 
quency domain. To be able to resolve very low- level 
phenomena, it must have a wide dynamic range and very 
low le\^els of distortion and spurious signals. The excep- 

*M you multipiy 16.384 toy 30 fn^. the re&uU as aotua'ty 492 ms. However, faf at teast 10 i^S 
oF eacfi ADC convefs^oni. iha HP 3456 A is noi measuring Tr>e Jnpui and provjd&s a TTL 
signal Indicating iJiis lact Ttiis time can be overlappecf with the DAC s settltng lima, tnereoy 
reducing the terta' acquisition Nme 
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tional DFT performance of tlie HP 3458A results from its 

combination of precise timing and the nearly ideal noise 
rejection capability of an integrating ADC. Also* its high- 
resolution track-and-hold circuit allows very fast sampling 
v\ith maximal time certainty. These features^ combined 
with the display capabilities of a host computer, are all 
that is needed to implement a high-resolution single-chan- 
nel oscilloscope or DFT analyzer. 
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A Structured Approach to Software Defect 
Analysis 

An effective software defect analysis requires that the 
relationships between program faults, human errors, and 
flaws in the design process be understood and 
characterized before corrective measures can be 
implemented. 

by Takeshi Nakajo, Katsuhiko Sasabuchi, and Tadashi Akiyama 



PROBLEMS THAT OCCUR IN SOFTWARE DEVEL- 
OPMENT because of human error negatively affect 
product quality and project productivity. To detect 
these problems as early as possible and prevent their recur- 
rence, one approach is to identify flaurs in present software 
development methodologies and procedures and recom- 
mend changes that will yield long- term defect prevention 
and process improvement. Typical approaches to software 
defect prevention have been to: 

a Investigate only design methodologies and procedures 
and then recommend such things as different languages 
or more tools as defect prevention measures^ 

■ Analyze the problems resulting from current design 
methodologies and procedures and develop solutions 
for each class of problem. 

The first approach is the most widely used and has 
tended not to he data-driven, thus making the investigation 
tedious and the results ambiguous. In contrast, the analysis 
of problems tends to produce less ambiguous results and 
data collection is easier, but it has typically been used only 
to solve immediate problems and therefore has produced 
only short-term solutions. 

To break out of the status quo, the instrument division 
of Yokogawa Hewlett-Packard [YHP] joined with Kume 
Laboratory of Tokyo University to analyze 523 software 
defects that occurred in three products developed by YHP. 
We tried to identify the flaws hiding in our current software 
design methodologies and procedures, and examine the 
impact of using the structured analysis and structured de- 
sign [SA/SD) methods. ^^^ This paper discusses the results 
of this joint investigation. 

Proiects Investigated 

The 523 software defects used for our investigation oc- 
curred during the development of three projects at YHP, 
which shall be called projects A, B, and C in this paper. 
Project A is a large all-software measurement system for 
analog-to-digital and digital-to-analog converters^ and proj- 
ects B and C are firmware for measurement instruments. 
360 defects were studied from project A and 163 defects 
from projects B and C, These software systems have the 
following common characteristics: 

■ They are intended to control hardware, that is, initializa- 
tion, setting registers, data retrieval, and so on. Therefore, 



they are greatly affected by the accuracy and clarity of 
hardware specifications. 

Their main parts are intrinsics, which are functions that 
can be used in measurement programs, or commands, 
which can be used sequentially to control devices. 
They are used to control hardware status, which means 
that they need many global variables to keep track of 
hardware states. 
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Fig. 1, Types of program f suits 
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Analyzing Software Defects 

Three types of inform ation useful for a defect analysis 
can be derived from a software defect: the human error (on 
the part of the developer), the program fauhs caused by 
the human error, and the flaws in the process causing tJie 
hmnan error. Human error is an unintended departure from 
work standards or plans. Program faults are outright errors 
in the software which result in the system's crashing* pro- 
ducing wrong results, and in general not behaving as 
specified. Flaws are imperfections in the design method- 
ologies or development procedures that affect the occur- 
rence rate of human errors and the possibility of detecting 
human errors before they become program faults. Examples 
of flaws include no documentation, confusing specifica- 
tions, nonstandard coding practices, bad methodolog>\ no 
inspections f poor test planning, and so on. 

To identif>' the flaws hiding in the design methodologies 
and procedures, we need to understand the mechanisms 
that cause human errors < and determine the relationship 
of these errors to program faults. This analysis is not easy 
because the human error process cannot be observed by 
objective methods, and usually, there isn't enough error 
data to analyze the relationship to program faults. However, 
the flaw^s must have some common factors, and they are 
reflected in the program faults caused by the human errors 
that occur during the design process. By design process 
we mean the portion of the softw^are life cycle devoted to 
the definition and design of a product*s features, soft^^are 
architecture, modules, and data structures. 

Program Faults 

To identify the types of faults that occur in programs, it 
is necessary to study what caused the problem and what 
corrections were made to fix the problem. Classification of 
faults based only on their outward appearance does not 
work welL Categories of faults such as "wrong range of 
loop counters in DO statements*' or "omission of conditions 
in IF statements*' define the coding problem, but they do 



Projecl A 
360 Faults 



Project B 
97 Faults 



Project C 
&6 Faults 




I (nterface [ | Function [ I Internal Process 



Size (KNCSS)' 



Pro^ B 
Project C 



Laf^guage 



Pascal. C 



Assembtv. Pascal 



*1&2 Matching Faults and 42 Reslriction FauUs 

*'KNCS5 - thousan<is of noncomment source statemerifs 

Fig. 2. Di^tnoutfon of ptogmm fauits for the fhrea projects tn 
this study 



not provide a clear correspondence between the fault and 
the design process. We still need to know^ the role of each 
program segment in the system. For instance, in the DO 
loop range problem, was the range error related to the 
number of hardware units, or the lengtb of the data file? 
Understanding program faults from the designer's point of 
view can help us link program faults to flaws in the design 
process. Fig. 1 shows our categorization of program faults 
along with examples of each catego^\^ Module interface 
faults relate to Iransferrmg data between modules, global 
variables, and hardw^are. Module function faults relate to 
a module's performing the wrong function. Module internal 
process faults correspond to logic errors, internal inconsis- 
tency, and programming rule violations. 

Based upon the program fault classification given in Fig. 
1, Fig. 2 shows the distribution of these faults among the 
three projects studied in this paper. The percentages of 
module interface faults and module function faults are 
similar for all three products (91%, 81%, and 85%). Since 
our design process was relatively the same for all three 
projects » we guessed that there must be some flaws in our 
design process associated with the ^vay we do module in- 
terface definitions and module function definitions. Since 
module internal process faults had the lowest frequency 
of occurrence and because these faults are more directly 
related to the coding phase, they were not given further 
analysis. 

Module Interface Faults. From Fig. 1. interface faults can 
be further classified into matching faults (mismatched data 
transfer between modules or hard ware) » and restriction 
faults (omission of checks on transferred data). The ratio 
of the number of matching fauhs to restriction faults turns 
out to be the same for all three projects and is about four 
to one. Consequently, we decided to focus our attention 
on matching faults for further study. Fig. 3 shows the five 
types of matching faults and their distribution for project 
A-* These five typf.s of matching faults are defined as fol- 
lows: 

■ Wrong correspondence between values of data and their 
meanings (e.g., storing the value r into a global variable 
that is supposed to contain the value r-1, or selecting 
the wrong destination hardware) 

■ Wrong data type, structure, or order [e.g., mismatch in 
the structure or order of arguments passed between pro* 
grams, or mismatch in the order of arguments read from 
a data file or a hardware interface) 

• Wrong correspondence betw^een names and their mean* 
'The same teuh ty&es artd va ry s^rmEai disf f iburtons w^fB timQovmwl tor pr oiecta B and C 

182 Faults 




Data Type 

and Siruclure 

29.7% 



Name 
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Data Value 

and Meaning 
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Rg, 3. Dfstrl button and types of modufe interface matching 
faufts for project A. 



APRIL imS HEWLETT-PACKARD JOURNAL 51 



)Copr. 1949-1998 Hewlett-Packard Co. 



ings (e.g., using the wrong argument m a calling se- 
quence , reading from the wrong global variabie, or setting 
the wrong hardware registers) 

■ Wrong method of processing data (e.g., omission of cer- 
tain steps when setting up hardware for some task such 
as a DMA transfer, or omission of initialization condi- 
tions or variables when calling other routines) 

■ Wrong name (e.g., using the wrong name to call a module 
or to access a global variable). 

Module Function Faults, Function faults are program faults 
resulting from a module^s performing the wrong internal 
operations. Fig. 4 shows the four types and the distribution 
of module function faults for project A.* These four types 
of function faults are defined as follows: 

■ Missing or unnecessary operations (e.g., failure to save 
calculated data to a global variablef or unnecessary cali- 
bration of hardware) 

■ Missing condition checks (e.g., saving data to a global 
variable before checking to see if it is permitted to save 
data to that particular variable) 

■ Wrong behavior of functions (e.g.. making the wrong 
decision, or calculating the wrong value because the 
wrong coefficients are used in an equation) 

■ Wrong order of functions [e.g., checking whether a 
hardware unit exists after setting it). 

The Design Process 

Our design process for instrument control software con- 
sists of the following steps: 

" Definition of unit functions and product features which 
are documented in the system external reference specifi- 
cations (ERS) 

■ Definition of data structures and module interfaces 

'The same (auiE types and very similar rt.stnbution^ we?re discovered lor projects B and C 
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Fig, 4, Distfibution and types of moduie function faults tor 
project A. 



which are documented in the internal reference specifi- 
cations (IRS) 

■ Coding of each module 

■ Iteration through the previous steps as necessary. 
Each of these steps includes the appropriate deliverables 

(specifications, test plans, etc.). and verification activities, 
such as design reviews and code inspections. Design re- 
views are done on the external and internal reference 
specifications, and code inspections are performed on 
selected modules. 

These documents and procedures are intended to ensure 
that a defect-free product is eventually produced. However, 
this goal cannot be attained if we do not have a clear knowl- 
edge of the types of human errors that occur in the design 
process and of the features of the documents and proce- 
dures that affect the error occurrence rate and error detec- 
tion. Consequently, we need to identify the types of human 
errors that cause program faults, the flaws in the present 
design documents and procedures, and the relationships 
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between them. From this perspective* we used the informa- 
tion gathered from investigating the two prevalent program 
fauJt t>^pes — module interface matching faults and module 
function faults — ^lo derive the human errors associated with 
each fault tvpe. These relationships were derived from inter- 
views with the design engineers and our own analysis. Fig. 
5 summarises the relationships between the two main types 
of program faults, human errors^ and flaws in the design 
process. 

Human Errors and Process Flaws 

Fig. 6 shows the distribution of the different types of 
human errors we discovered during our analysis. The 
human error that caused each software defecl was not al- 
ways clearly recarded. However, as we did for deriving the 
information in Fig. 5, we analyzed various documents and 
inter%^iewed the designers and programmers who de- 
veloped the system to come up w^ith the numbers and per- 
centages shown in Fig. 6, 

Human Errors and Matching Faults* The human errors 
responsible for causing module interface matching faults 
are defined as follows: 

■ Module or Variable Specifications. Module interfaces 
and global variable definitions are missing or misun- 
derstood. 

■ Hardware Specifications. Software developers overlook 
and misinterpret hardware specifications or other tech- 
nical requirements of the hardware. 

■ Design Changes. Changes to the hardware interfaces, 
other related systems, or module interfaces are not com- 
municated properly. 

■ Related System Requirements. Technical requirements 
are not communicated clearly between development 
groups and other related systems. 

As shown in Fig, 6a. human errors associated with 
hardware interface specifications and module interfaces 
were the most frequent. Therefore, we examined the design 
process and found the following flaws associated with 
these error types. 

■ Hardware Specifications. Hardware specifications are 
difficult to read and understand for software engineers, 
and as a results some important technical requirements 
about their interfaces were omitted. This flaw affected 
our externa! and internal design steps. We found that 
hardware interface information for writing softw^are driv- 
ers was being derived from circuit diagrams, which were 
difficult for software developers to use without error. 



■ Module or Variable SpeciOcatiQns. The results of defin- 
ing interfaces in the lower- level modules were not well- 
documented before defining internal algonthms and 
coding modules. Therefore, it was difficult to find mod- 
ule interface mismatching faults in design reviews. There 
was a lack of uniformity in the definition of certain fea- 
tures* and complicated interfaces between modules were 
not taken into consideration. These flaws also affect our 
internal design activities. 

Human Errors and Function Faults. The human errors re- 
sponsible for causing module function faults are defined 
as follows: 

■ Module Specifications. Errors !b the translation from 
external specification to internal module specifications 
or misunderstanding of module specifications, 

■ Commands and Intrinsic Specifications. Misunderstand- 
ing the system external specification and providing the 
wrong or incomplete functionality for system features. 

■ Status Transition. .Missing or misunderstanding the val- 
ues of global variables that define the different state 
transitions of the system or hardware. 

■ Related System Requirements, Missing or misunder- 
standing the technical requirements of other related sys- 
tems or hardware, resulting in such mishaps as the use 
of the wrong Information from another subprogram to 
set hardware. 

As shown in Fig. Gb, human errors associated with com- 
mands, instrinsics, and module functions were the most 
frequent. Therefore^ we examined the design process and 
found the following flaws associated vvifh these error types. 

■ Commands and Intriusics. During the first part of our 
design process, when the external specification is de- 
fined, the independence between the functions of the 
commands and intrinsics was not sufficiently defined. 
For example, functions associated with the user interface 
were not partitioned properly, resulting in overlap in 
functionality, thereby causing program faults. Another 
problem was that the external specification documenting 
the commands, intrinsics. and other system require- 
ments was not systematic* The specifications were 
mainly written in natural languages, which resulted in 
ambiguity regarding the uses and functions of commands 
and intrinsics* 

■ Module functions. During the internal design phase of 
our design process, when the modules and data struc- 
tures are defined, developers designed the module struc- 
tures based mainly on considerations about system per- 
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formance. This resulted in modules that had no clear 
correspondence with the system external specification. 
Another problem was that module functions were not 
completely specified before the internal algorithm and 
coding of each module were started. Internal design 
specifications also suffered from a lack of systematic 
tiocu mentation, resulting in ambiguoiis module func- 
Hons. 

Design Process Issues 

In the previous section we determined the flaws in our 
design process that caused the human errors resulting in 
program faults in our productSn Based upon wliat we 
learned about these flaws, three issues were derived from 
our analysis. Fig. 7 shows an example of the relationship 
between these issues and our design process. 

Issue 1. A systematic method is needed to translate sys- 
tem features defined during product investigation into the 
details of a clear system external reference specification. 

Issue 2. A systematic method is needed to translate exter- 
nal specifications into module structure and module func- 
tions. 

Issue 3, A systematic method is needed to specify the 
techniciil requirements of hardware and to translate these 
requirements into software module interface specif Ica- 
tioiis. 



The above issues are vital to our design process. Since 
most of our products have similar characteristics, any solu- 
tions to these Issues would pertain to all our software prod- 
ucts. Issues 1 and 2 indicate that we need a method to 
translate from one level of abstraction to another^ with each 
translation making it easier to perform a systematic en- 
gineering analysis of the system. With a good analysis 
methodology we can check the independence and suffi- 
ciency of functions and review their specifications to find 
unsuitable function definitions. Issue 3 requires that we 
have a met hodu logy that enables hardware engineers to 
communicate hardware interfaces effectively to software 
engineers, and enables software engineers to communicate 
module interfaces and other system interfaces among them- 
selves. With such a methodology, module structure and 
design can be effectively reviewed by tha hardware and 
software engineers of the design team as well as those who 
must test and support the product. 

SA/SD and Design Process Issues 

Our uivestigalion hd us lo believe that the structured 
analysis and structured design (SA/SD) methodology is the 
most suitable candidate for dealing with the three design 
process issues. We believe that SA/SD design methods can 
help prevent program fauhs by enabling us to detect and 
correct these problems before they become real defects. 




^H Some part of Ihe comparator function is in 
th« interpreter module. 

I -^1 HP-IB comparator commands are located in 
interpreter and comparator modules. 
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Fig. 8 shows the correspondence between the three design 
issues and the soiutions offered by S.VSD methods. 
Proposed Soltitioti for Issue 1. The key elements of struc- 
tured analysis we found useful for dealing with issue 1 
include: 

■ Context diagrams, which define the relationship be- 
tween the software system and its environment [e.g,, 
relationship between the hardwmre and the firmware ele- 
ments in an instrument) 

■ Data flow diagrams, which define the actions of pro- 
cesses (modules or functions) in the system, and the data 
and control flo%vs between these modules 

■ Process specifications, which define the functions and 
behavior of the processes in a precise structured lan- 
guage. 

The functions we define for otir syistems are organized 
based on their relationship with data. The functions that 
depend on each other are difficult to classify into simple 
groups. In structured analysis, detailed functions of each 
intrinsic or command and their relationships can be rep- 
resented by data flow diagrams. Also, the data relationships 
are clearly specified, and the operation of each function is 
defined in a structured language in the process specifica- 



tion. 

Proposed Solution for Issue 2- The system external specifi- 
cation can be smoothly translated to the module structure 
by using the transformation analysis technique provided 
by SA/'SD, Transformation analysis enables us to take each 
data flow diagram and transform it into more detailed data 
flow diagrams or module structure. By applying this 
methods we can make a module stmcture that has a clear 
correspondence to the system external specification. 
Proposed Solution for Issue 3. The key elements of struc- 
tured design we found useful for dealing with issue 3 in- 
clude: 

■ Structure charts, w^hich define the module hierarchy 
within a data flow" diagram or within a module 

■ Module specifications, w^hich define in a structured lan- 
guage the function of each module 

■ Data dictionaries, which define the data that flows be- 
tw^een modules. 

Among these elements, the data dictionary provides us 
with the greatest leverage to solve issue 3. With the data 
dictionary we can systematicBlly specify the interfaces to 
the hardware and the interfaces between the software mod- 
ules. With these interfaces consistently defined we can 
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easily detect misinatches between modules and hardware. 

Conclusion 

In this invest igation. we tried to identify the flaws hiding 
in our current software design methodology and proce- 
dures and examine possible countermeasures against them. 
VVe analyzed about five hundred actual problems that oc- 
r:urred during soft^vare development for three instruments 
and used these defects as a basis for our investigation. 

We believe that SA/SD methods can solve some of our 
design problems. However, there are still some challenges , 
which include: 

■ Elimination of the inconsistencies between the present 
specifications using natural languages and the new 
specifications using the SA/SD methods 

■ Installation of automated tools for using the SA/SD 
methods 

■ Establishment of an appropriate education and training 
system on the SA/SD methods for the softw^are engineers 

■ Preparation of other groups m our division for dealing 
with documents written using SA/SD methods 

■ Establishment of design review methods based on the 



SA/SD methods 

■ Investigation and use of other tools and techniques pro- 
vided by SA/SU, such as state transition diagrams and 
transactional analysis 

■ hivestigatlon to find ways to model software behavior 
that cannot be analyzed with current SA/SD methods. 
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Dissecting Software Failures 

Beyond collecting software defect data just to study defect 
frequency, this paper outlines a quality data collection 
process, an effective analysis process, and a method to 
justify changes in the software development process based 
on the defect analysis. 

by Robert B. Grady 



MOST PEOPLE DONT LIKE TO BE TOLD that 
they've made a mistake. It*s only human not to 
want to be wrong. On the other hand, software engi- 
neers don't intentionally make mistakes, so if we can under- 
stand why mistakes occur without accusing indi%^i duals, 
we might eliminate the causes of those mistakes. Unfortu- 
nately, discussions concerning software defects are confus- 
ing because different people describe them from different 
perspectives. 

This paper discusses some of the terminology of these 
different views. It then examines some simple data collec- 
tion and analysis techniques that help identify causes of 
defects and point to areas where improvements can be 
made. Finally, it presents some guidelines for justifying 
change based upon the results of analyses. 

"A defect is any flaw in the specification, design, or 
implementation of a product."' Such flaws cause managers 
to lose control by reducing their ability to predict when 
development or maintenance will be completed. Thus, we 
encounter another human trait: people like to be in control 
of a situation. The opportunity. tbeUn is for software de- 
velopers and managers to record sufficient defect data 
while analyzing and resolving defects to understand and 
remove the causes of those defects. 

Defect Perspectives 

Fig. 1 illustrates three views of a defect. Each of these 
views is c:haracierii'.ed by its own terniinolo(^y and focus. 
When users of a product have a problem, they know that 
they can't get their job done because the software product 
isn't performing the way they expect it to. The level of 
their concern reflects how much their business is impacted, 
and terms like critical or serious mean that they stand to 
lose subslanlial time and/or money if something isn't done 
soon. 

On the other hand, individuals responsible for com- 
municating defect information and status to and from cus- 
tomers refer to which component is at fault, whether a 
patch exists, and when a permanent fix can be expected. 
They must extract enough detail from customers to discover 
workarounds and to provide maintainers enough informa- 
tion to seek a permanent fix. 

The third perspective is that of the people responsible 
for maintaining and enhancing software. They speak in 
terms of what code was at faulty the priority associated 
with correcting the defect, how difficult il will he to fix, 
and when to expect the fix. 



If we draw an analogy to medicine, the patient describes 
the problem in terms of what hurts and how much. The 
nurse setting up the appointment must ask enough ques- 
tions to tell the doctor enough to form preliminary conclu- 
sions and to determine how urgent it is for the doctor to 
see the patient. The doctor must run tests to discover the 
real cause of the ailment and must prescribe the correct 
treatment to heal the patient. 

Data Collection 

1 he first step in treating software defects is data collec- 
tion. Most organizations already gather much of the neces- 
sary' data. What Is proposed Is a method to use the data for 
a long-term quality improvement effort, not just to solve 
the current problems. 

For example, there is always justifiable pressure to fix 
urgent problems. The goal is to maximize customer satisfac- 
tion (with an emphasis un timeliness, in I his case). In pur- 
.suit of that goal, data is colletited to optimise the flow of 
information from the customer about a defect [see Fig. 2). 
Additional data is collected as engineers investigate the 
problem to provide information to the customer regarding 
status and possible fixes. Once customer satisfaction is 
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achieved and the customer has a workaround or permanent 
fjx for the problem, data collection should nol stop. 

ff we want to learn from past mistakes lo improve de- 
velopment or support practices, then a small additional 
amount of time must be spent to collect additionaj data. 
What are some of the questions that this long4erm goal 
prompts, and what data is ne^eded to answer the questions? 
Some of the more obvious questions are: 

1. What development or maintenance process failed? 

2. Hovt^ often do such failures occur? 

3. Htnv expensive is it to fix such failures? 

4. Whk:h components are most subject to failure? 

5. What process change will detect or eliminate these 
failures? 

Fig, 3 shows an example of the additional data needed 
for the defect described in Fig. 2. The numbers in Fig. 3 
are related to the questions above. Question 2 could only 
be answered by analyzing the defect type information for 
a number of similar defect fix reports. 

Resistance to data collection when defects are being fixed 
is natural, because there may be a bficklog of defects and 
strong schedule pressures. A common request is for addi- 
tional automation aids to capture the information suggested 
in Fig. 3^ and unlil then, uu data is collected. Such requests 
sainetinies miss the poinl, however, and fail into the trap 
of what we'll call the ''automation syndrome." In fact it is 
unlikely that entry of such data into an automated system 
would shorten the time involved on the part of the en- 
gineers reporting It. 'I be problem with the automat itni syn- 
drome is that it caEi prevent the collecllou of needed data 
for years if simple solutions and firm management don't 
prevail. 

We need to ask what it costs (in time) to collect this 
additional data, Let's take a typical product of 100 KNCSS 
[thousands of noncomment source statements). We have 
seen average postrelease defect densities from less than 0.1 
to as high as l or 2 defects per KNCSS in the first year after 
release of a product, For the sake of calculations, let us 
assume a very high value of 1 defect. KNGSS. For our prod- 
uctt dien^ we w^ould expect 100 defects in the first year. If 
we look at the data requested in Fig. 3, it seems likely that 
it would lake betvveen five and ten minutes per defect to 
provide the fix information requested in Fig. 3. This means 
that the total incremental engineering hours for our defect- 
plagued 100- KNCSS product might vary from one to 
slightly over two days (see Fig. 4]. Not a very large invest- 
ment for such valuable data. 



Suppose the product that you are going to perform post- 
release analysis on is ten times as large, or that you want 
lo perform your analysis before product release (where we 
typically see about ten times the number of defects as in 
postrelease). Data collection time is always a sensitive 
issue, so yon should consider how to capture the informa- 
tion that you need while minimizing tlie data collection 
time. This i.^ done by collecting sufficient samples of defect 
data to yield an effective distribution of causes. The total 
will be adequate as long as the sample size is large enough 
(probably 100 to 150 samples] and sufficiently random. 
For example, you might restrict the capture of I he addi- 
tional data shown in Fig. 3 to just critical and serious de- 
fects, or turn selection into a game by rolling dice before 
starting work on each defect. 

The goal of the data collection scheme is to optimise the 
amount and quality of information and to minimize the 
time [cost) spent by those supplying the information, 
whether the collection method is manual or automated. 

Data Validation 

When you initiate the collection of a new set of data, 
adjustments are needed in people's activities and existing 
processes. It is particularly Important at the start to include 
procedures to ensure valid data. 

A common cause of invalid data is differenl inlerpreta- 
tions of definitions. These are somewhat alleviated by 
proper training before collection begijis, but all loo often 
we incorrectly assume that everyone will interpret instruc- 
tions in the same way. For example^ two different liP divi- 
sions reported that many defects labeled as coding defects 
were really caused by changed requirements or design. The 
incorrect labeling occurred because the defects were discov- 
ered or fixed during the coding phase. 

It is desirable to reinlorce iniliai collection u\ detect in- 
formation with subsequent interviews. These should be 
initiated by the project manager or a representative to en- 
sure that the data is accurate and to emphasize the impor- 
tance of accuracy to the engineers reporting the data. These 
checks should examine a IcUge cross section of the data in 
depth. Once this is accomplished, spot checks are probably 
all thai are needed to maintain the flow of good data. 

Data Analysis 
In the previous sections, the iocus was on cpllectipn of 



Fixed by: Lynn Smith 
Date fixed ^ 9^9 84 



Submitter: Bruce Davis Date SubmiHed: 8 22 84 

Company Name: Hewlett-Packard Dept: SEL 

Support Engineer L John Michaels Support Office: Factory 

Computer System Model; 3000 930 Identification No.j 0-13-821844? 

Defective Software; MPE-XL Release Version: MIT XB 6. 06 
Seventy {Critical, SeriousK 

Medium, Low]: Sercous 

Workaround? (Y H)i Y (easydifffcuft)?- Difficult 

Symptoms: System crashes with abort number of 1072. This has 
happened twice in the last week. At the time of the crash, 
there was one user on the system, mgr. official, The job running 
was newjobxL 

Fig. 2. Simplified defoct report. 



® Engineering Hours lo Ffnd and Fix: 86 
Defect Origin: Design 
5^ Defect Type: Data Definition 
Category of Defect 

(Missing^ Unclear, Wrong. Chariged, Other] i Wrong 

® l^odules Changed: Di&c.ta. Table 5 
Other modules affected: inters 

(f; How could defect have been found earlier: 

Design walkthrough: more com pSeie test coverage: 
more iimely data dictionary updates. 

Fig. 3. Defect tm mtormaUon The numbers refef to the ques- 

tions <n me article 
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valid data. The next step of the process is the analysis of 

the data collected. Again, there is a danger that nothing 
will happen, because many managers hav^e never taken the 
time to perform such an analysis. They believe that the 
tme Involved will be too great. Perhaps this belief is as 
unfounded as the one concerning the data collection time. 

What are the steps involved in a typical analysis? The 
foliowing estimates assume that the analysis is begun with 
100 one-page completed defect reports and is done manu- 
ally. 

1. Sort the data collection forms by defect origin. Count 
the number in each group and total the number of engineer- 
ing hours to fix the defects for each group. Arrange the 
totals in descending order of total engineering hours (30 
min), 

2. Calculate the average fix time for each of the totals from 
step 1 (5 min), 

3. For the top two or three totals in .step 1 , count the defects 
sorted by defect type and multiply by the appropriate av- 
erage fix times. Limit the number of t>^pes to the largest 
totals plus a single total for all others (In min). 

4. Add up the defects sorted by module changed. Limit 
the number of choices to the five most frequent plus a 
single total for all others (15 min), 

5. Review the defect reports for the defects included in 
the largest totals from siteps 3 and 4, and summarize the 
suggestions for haw the defects could have been found 
earlier (1 hour). 

Following the procedure aboven project managers would 
know se%^eral valuable fmX^ after only about two hours 
time. They would know what the most costly defects were, 
when they occurred, where ihey occurred, and the most 
likely steps to take to prevent their occurrence in the future. 

But even two hours of a project manager's time is some- 
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times difficult to find. Other useful alternatives that have 
been successfully tried are to use engineers from a quality 
or metrics organization or to hire a student from a locaJ 
university to perform the analysis. 

A Model for Analyzing Defect Causes 

Various reports have documented successful efforts to 
analyzti defects, their causes, and proposed solutions.^"^^ 
But the terminology among them has differed considerably, 
and the definitions could possibly mean different things 
to different people. In the fall of 1986 the HP Software 
Metrics Council addressed the definiticm nf standard 
categories of defect causes. Our goal was to provide stan- 
dard terminology for defects that different HP projects and 
labs could use to report, analyze* and focus efforts to elimi- 
nate defects and their causes. 

FortuiiateiVr the lElvE had a subcommittee working on a 
standard for defect classifh:ationJ^ so it was possible to 
start from their working documents. The IEEE definitions 
covered all phases of defect tracking in an extensive general 
way. These will undoubtedly be of value to the people 
supporting defect tracking systems. Unfortunately, the 
IEEE document at that time was ver\- Jong and too general 
to be applied specifically to any project. As a result, the 
metrics council extracted only the material related to defect 
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causes and produced a metrics guideline that is easier to 
use.^* Fig. 5 illustrates a model of defect sources taken 
from the guideline, and the box on page 6Z gives the defi- 
nitions from the guideline. 

The model is used by selecting one descriptor each from 
origins, t>^pes, and modes for each defect report as it is 
resolved. For example, a defect might be 3 design defect 
where part of the user intHrface described in the inlernni] 
specification is missing. Another defect might be a coding 
defect where some logic is wrong. 

An Example 

Let us look at a specific example using the model pre- 
sented in Fig. s. The data for this example is laken from a 
detailed study of defect causes done at HP.^^ In the study, 
defect data was gathered after letting began. Fig, fi shows 
the data sorted by the primary origins of defects. 

It is desirabie to focuK attention on the causes of defects 
that cost the mo.st to fix. The nel co.st of any given classifi- 
cation is represented by the total defects for the classifica- 
tion times the average cost to fix those defects. This study 
didn t accurately record the engineering times to fix the 
defects, so we will use average times summarized from 
several other studies to weight the defect origins. ^^ In par- 
ticular * the average engineering cost to fix coding defects 
that are not found until lesling is abou! 2.^ limes the cost 
of those found during c:pding. The factors for design and 
specification defects that are not found until testing are 
about 6.25 and 1 4. 25 , respectively. Fig. 7 shows the relative 
costs to fix the defect population from Fig. B when the 
weighting factors are applied- For the sake of this example, 
the other origins are assumed to have a multiplier of one, 
and we will normalize all fix times to assume that a coding 
defect fixed during coding takes one hour. The weighting 
factors then simply become engineering hours to fix the 
various defect categories. 

These two figures illustrate step 1 of the five-step proce- 
dure described earlier and the weighting factors in Fig. 7 
represent step 2. The study from which this data was taken 
only provided defect type data for coding and design de- 
fects. Ttierefore, we will perform step 3 of our procedure 
with a breakdown of data for only coding and design. This 
is shown in Fig. 8, It suggests that efforts should be focused 
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Fig. 7. Weighted dislrihution of defect origins. 

on eliminating logic errors, computation errors, and pro- 
cess communication errors before final test. 

These brief examples show how easy it is to apply the 
analysis procedure to discover where changes with the 
greatest impact can be made. They also show how incom- 
plete data can force us to make assumptions that might 
impact the accuracy of our conclusions. In this example 
we didn't have complete data regarding specifications de- 
fects or data detailing engineering hours for all defects. 
These are probably not serious drawbacks in this case, but 
one must be certain to identify such uncertainties and their 
potential effects every time an analysis is performed. 

Here is an interesting note io conclude our example. The 
use of weighting factors in the analysis above emphasized 
working on eliminating causes of problems that cost the 
most to resolve. The assumption was that we would insti- 
tute process changes to eliminate the causes of those de- 
fects. An excellent source of these changes would be the 
suggestions collected when defects are resolved. If the em- 
phasis is to reduce the defect backlog tis quickly as pos.sible, 
then the effort must be focused on those problems that are 
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easiest to fix quickly. In that case, we would simply view 
the data differently to learn the answer. We would look for 
defect groups consisting of large numbers of defects that 
can be resolved quickly (e.g.. documentation or some t\iDes 
of coding defects)- 

Justifying Change 

Once you have collected the data necessan^ to under- 
stand w^hich defects impact your operation and analyzed 
the data to determine what youi tactics should be, you 
encounter the most difficult step. This step is to recom- 
mend and implement change based upon the discovered 
facts. It IS the most difficult because of another human 
characteristic: resistance to change. There axe many facets 
to such resistance that span the entire implementation pro- 
cess. These are discussed in detail elsewhere,^ ^ so this 
paper will focus on the first step in the change process 
only, that of initial juslificallon of change. Recommenda- 
tions for change take many forms, but most successful 
changes are based upon a cosi^enefit analysis built from 
components such as those outlined in Fig. 9. 

Most of the entries in the benefit column of an actual 
table would be represented in measurable units based on 
data already collected and analysed using the techniques 
described eaxlier. The remaining items would be estimates. 
The entire table should be bound to a specific time period, 
such as one year. A summary table can be constructed from 
an appropriate subset of items, costs, and benefits given in 
Fig. 9 that should be convincing by itselL F'or extra em- 
phasis* it can be supplemented by benefits beyond a year 
and by more-difficull-to-measure customer satisfaction 
benefits and increased sales. 

An Example 

LuTs build a case for change based on data from two 
studies done at HP. The first study investigated the number 
of engineering bours used to find and fix differeni defect 
types primarily during implementation and tesl.^'' It found; 

Average number of design defects = 18% of total defects. 

The second study evaluated various factors related tn 
design and code inspections.^'^ It found: 

■ Optimum number of inspectors = 4 to 5 

m Ratio of preparation to inspection time > 1.75 

■ Inspection rate = 300 to 400 lines of design text/hour 

■ Average number of defects found per inspection hour 
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Fig* 10. Results of ussng in formation from several studies to 
show a justification for design inspections a) Analysis of times 

to find design defects bj SampSe cost /bene fit analysis of 
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= 2.5. 

Fig. 10 shows the results of combining the information 
from these two studies into a Justification of design inspec- 
tions for a 100-KNCSS project. 

Note that neither costs nor benefits were specified for 
the item tnanagement. For simplification we assume that 
roughly the same management time will be needed to intro- 
duce the new concepts to a project team as would ha\'^^e 
normally been needed to manage the additional engineer- 
ing hours using the old techniques- 

In summary, the introduction of design reviews seems 
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Defect Origins and Types 



Enhance meat. A change that could not possibly have been de- 
tected, or, if detected, would not have been corrected. 

An enhancement is not a defect. Restraint must be exercised 
when a software change is labeled as an enhancement The use 
of Ihe term enhancement should be restricted to those cases 

where the customer's needs and /or the product scope have truly 
changed since the release of the product, thereby creating new 

requirements that could not have been anticipated in the original 
deveiopmant effort For example, the performance of a software 
product was competitive upon release, but it needed to be im- 
proved two years later to remain competitive. Such a change is 
an enhancement. If the performance was not competitive at the 
original time of release, then any subsequent change to improve 
performance is cor^sidered a defect fix. 
Specification Defect. A mistake in a specification that sets forth 
the requirements for a system or system component Such mis- 
takes can be in functional requirements, performance require- 
ments, interface requtrements, design requirements, develop- 
ment standards, etc 

■ Requirements or Specifications. The specifications do not 
adequately describe the needs of target users. Also includes 
the effects of product strategy redirection and nonexistent 
product specifications. 

■ FuncttonaliEy Problems with the product feature set (e.g.^ in- 
correct or incompatible features). Includes cases where func- 
tionality is increased to add market vaiue. 

■ Hardware, Software, and User Interface. Problems wtth tncor- 
rect specification of how the product wiJl interact with its envi- 
ronment and/or users. 

■ Functional Description. Incorrect description of what the prod- 
uct does. Generally discovered during requirements or design 
inspection. 

Design Defect A mistake in the design of a system or system 
component. Such mistakes can be in system or component aJ- 
gorithms, control Jogic, data structures, data set use information., 
input/output formats, and interface descriptions. 

■ Hardware, Software, and User Interface. Problems with incor- 
rect design of how the product wril interact wUh its environment 
and/or users. For example, incorrect use of libraries, desjgn 
does not implement requirements, device capabilities over- 
looked or unused, or design does not meet usability goals. 

■ Functional Description. Design does not ettectivety convey 
what the intended module or product should do. Generally a 
detect found during design inspection or during implementa- 
tion (coding). 

■ Process or Interprocess Communications. Problems with the 
interfaces and communications between processes within the 
product. 

■ Data Definition. Incorrect design of the data structures to be 
used in the module/product. 

■ Module Design. Problems with the control (logic) flow and 



execution within processes. 

■ Logic Description. Design is incorrect in conveying the in- 
tended algorithm or logic flow. Generally a defect found during 
design inspection or implementation. 

■ Error Checking, Incorrect error condition checking. 

■ Standards. Design does not adhere to locally accepted design 
standards. 

Code Defei^t. A mistake in entry of a computer program into the 
symbolic form that can be accepted by a processor 

■ Logic. Forgotten cases or steps, duplicate logic, extreme con- 
ditions neglected, unnecessary function, or misinterpretation 
errors. 

■ Computation Problems, Equation insufficient or incorrect, pre- 
cision Eoss, or sign convention fault. 

■ Data Handling Problems. Initialized data incorrectly, accessed 
or stored data incorrectly, scaling or units of data jncorrect, 
dimensioned data incorrectly, or scope of data incorrect. 

m Module interface/Implementation. Problems related to the call- 
ing of, parameter definition of, and termination of subprocess- 
es. For instance, incorrect number of, or order of, subroutine 
arguments, ambiguous termination value for a function, or data 
types incorrect 

■ Comments. Insufficient or incorrect commenting 

■ Standards. Code does not adhere to locally accepted coding 
standard. 

■ Miscellaneous (other); This classification should be used spar- 
ingly, and when it is used, the defect should be very carefully 
and extensively described in associated documentation, 

Docurnentalion Defect. A mistake in any documentation related 
to the product software, except in requirements specification 
documents, design documents, or code listings. Mistakes in the 
latter three are assumed to be specifications defects, design 
defects, and coding defects, respectively. 
Operator Defect: Any situation that involves the operator's mis- 
understanding of procedures, hitting the wrong button, entenng 
the wrong input, etc Does not necessarily imply that the product 
is in error. 

Environmental Support Defect. Defects that arise as a result of 
the system development and/or testing environment. 

■ Test Software. Problems in software used to test the product 
software's capabilities. For example, another application pro- 
gram, tf'te operating system, or simulation software. 

■ Test Hardware. Problems with the hardware used to run the 
test software, not the hardware on whtcb the product software 
runs, 

■ Development Tools. Problems that are a result of development 
tools not behaving according to specification or in a predict- 
able manner. 

Other. This classification should be used spanngly, and when 
it is used, the defect should be very carefully and e^^tens^vely 
described in associated documentation. 
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to be very desirable. Tbe beoefits in this example tolaJiy 
overwhelm the costs . so why aren't inspectloDs more 
widely used today? It gets back to the issue of resistance 
to change. Remember that while this example is based on 
real data, it is suspect since the data was measured hy 
someone else and is derived from several sources. When 
yon just% change, you most organise your arguments as 
clearly and persuasively as possible, and you must be pre- 
pared to continne trying to persuade the people involved 
onti] the change has occurred. 

The example was selected to illustrate the process of 
justifying change. The core of the justification was the data 
recorded in previous studies of defects and the times taken 
to resolve them. You can use such published data to help 
guide your decisions, but ultimately you must also collect 
enough data that is specific to your process or products to 
verify that the problems you pursue are the most important 
ones. 

Conclusion 

Mana^^ers of software development cannot afford to con- 
tinue producing nnd supporting products with the same 
old techniques and processes. The field is changing rapidly, 
and improvements in both quality and productivity are 
necessary to remain competitive. The history of the appli- 
cation of software metrics includes the continuous applica- 
tion of basic scientific methods. We collect data and estab- 
lish hypotheses for improvements. We take additional mea- 
surements to prove or disprove the hypotheses. And we 
revise our hypotheses accordingly and start the process 
again. The major problem of management without the use 
of data Is that the hypotheses can never be really vail dated 
and institutionalized. 

If we return to our medical analogy, it is like medical 
doctors having to practice medicine without understanding 
the human body through dissections and autopsies. For 
over a thousand years before the fifteenth century, medical 
doctors were prevented from dissecting human bodies be- 
cause of fear and superstition. W'hen the rules against dis- 
sections were eased, great progress occ:urred in a relatively 
short time. We must experience a similar renaissance 
period in software development. Perhaps it is time that our 
schools began to teach '^software autopsies/* 

The techniques described here for collecting, analyzing, 
and presenting data are simple, yet effective means to im- 
prove software development. We saw that the coUection 
of a small amount of additional data can yield a large 
payback in terms of useful information thai fits into a stan- 
dard framework for analysis, A five-step process for data 
analysis was given that organizes this Information to point 
to areas and methods for improvement. And a framework 
for justifying change to both management and engineers 
suggests how changes are proposed initially and justified. 
What rf^mains is for managers to use ihese tec:hniquQs as 
quickly as possible to promote positive change. 



Acknowledgments 
The members of the HP Software Metrics Council who 

particularly contributed to the Defect Categorization 
Guidelines were Dave Classick* Craig Fuget. Mike Gourlay» 
Bob Horenstein. Chuck Leath, and Kathy Osborne* I would 
also like to thank fan Grady. Debbie Caswell. Ken Hoyt. 
Barbara Noble. Danny Lau, and Sally Dudley for their help- 
ful suggestions regarding this article's development. 

References 

1 R. Grady and D. Caswell, Software Metrics; Establishing a Citm- 

pany-VVide Program. Frentice-HalL Inc., 1987. p, 224. 

2. M. L* Shoomsn. "Types, Distribution, and Test and Correction 

Times for Programming Errors/' IEEE Proceedings of the 1975 

Conference on BeUable Software, Los Angeles, Calif.. April 197S» 

pp. 347-357. 

:i. A. Endres, "An Analysis of Errors ami Th*jjr Causes in System 

Programs." IEEE Tran suctions on Software Engineering. Vol. SE-1 » 

no. 2, June 1975. pp. 140-149. 

4. R. J. Rubcjy. j. A. Dana, and P. W. Biche. "Quant ilalive Aspects 
of Softvvare V^aiidation," IEEE Transoctions on Software Engine«r- 
ing. Vol. SE-1. no. 2, June 1975, pp. 150-155. 

5. S, Boehm. J. R. Brown. H. Kaspar, M. Lipovv. G. MacLeod » and 
M. J. .Merritt* "Characteristics of Software Quality," TRW Series 
of Software? Technriiogy. Vol 1 . Amsterdam: TRW and Nforlh-Hol- 
land Publishing Company, lli78. 

6. N.F. Schneideivind and H.M. Hoffniari, "At} E?vpfctrimentiu Soft- 
ware Error Data CaHoction and Analysis.'^ IEEE Tranmctions on 
Software Engineering. Vol. SE-5. no. 3. May 1979, pp. 276-286. 

7. C. Sieloff. ''Software TQC: Improving Iht* Software Develop- 
ment Pratiess Through Statistical Quality Control,'' HP Software 
ProdLtcth'Ity Cnnftij^ncti Frocfuidinga. April 19B4. pp. 2-49 to 2-62, 

8. G, HamiltooH "Improving Software Developnii^nt Using Quality 
Control," HP Software Produf;!) vity Con/erente Proceedings, April 
IMS, pp. l^OG to 1-102. 

9. D. Kenyon, 'Implementing a Software Metrics PrDgram/" HP 
Software ProdLTcfivrty Conference ProaeedingB, April 1985, pp. 
1-103 lo M17. 

ID. C. Fuget, "Using Qual ity Metrics to Improve UfB Cycle Produce 
tivily. " HP SoftwarG Produclfvity Conference Proceedings, April 

tyafi. pp. i-a6 to 1-93. 

11. C Leat h , " A Sohw are Def fict A n a i ys i s , ' * I JP Soft ivcj re Prod u c- 
Hvity Conference Proceeding*;. April 1987. pp. 4-147 to 4- 161. 

12. A Stuiidard for Software Errors. Faults, unci Fuliures* IEEE 
working group Pi 044. March 19^7. 

13. " S o f Uv rj r R 11 tf v *:! ] t F J 3 m e n t Met r i t: s G u i d elf n e: De f ect A tia ly si s , " 
June 1987. jint«rnal HP memorandum). 

14. 8. Baehm, Softwure Engineering Economics. Prentrce-Hall, 
Inc., 1981. p, 40. Reprinted by permis,sjon of PrenliEie-Hall. \nc. 

15. R. Grady and D. Caswell, op, cit.. pp. B2-95. 

16. R, Grady. "Measuring antl Managing Software Mainlenance/* 
IEEE Software, September 194*7. pp. 35-45. 

17. B. Scott and 0, Decot, 'Inspection!; at DSD— Automating Data 
Input and Data Analysis.' HP Software Produf:IIWfy Confermice 
Pnntmiings, April 1985, pp. 1-79 to 1-89. 

18. R. Grady and IX Caswell, op. cif., p. 145. 

19. C, Jones, Progrommfn^ Produciivii\\ McGraw-Hill fionk Ccl, 
1986, p. 179, 



Apnn 1989 HEWLEH- PACKARD JOURNAL 63 



)Copr. 1949-1998 Hewlett-Packard Co. 



Software Defect Prevention Using 
l\/lcCabe's Complexity Metric 

HP's Wattham Division has started to use this methodology 
and its associated tools to catch defect prone software 
modules early and to assist in the testing process. 

by William T. Ward 



IT rS POSSIBLE TO STUDY. MEASURE, AND QUAN- 
TIFY many aspects of the software development procesSt 
and if sufficient data about good practices used in recently 
released projects is availabie, real-time adjustments can be 
made to ongoing projects to minimize past mistakes and 
to leverage ideas from past successes. 

HP's Waltham Division has maintained an extensive soft- 
ware quality metrics data base for products developed here 
over the past three years. We have been able to use this 
data base daring project postmortem studies to provide 
insight into the strengths and weaknesses of Waltham 's 
software development process. 

Fig. 1 lists the basic software quality metrics for two 
major Waltham Division products that have been released 
within the past two years. Based on the extensive tjmount 
of code, both of these products can be classified as large- 
scale firmware projects. These projects had a short develop- 
ment time and a very iow^ postrelease defect density. Since 
these products are considered technical successes, if was 
suggested that the software development data we had about 
them could be studied to improve our understanding of 
Walthaoi's software development process. This resulted in 
a formal effort to examine the project data in more detail. 

A substantial amount of software process data was 
evaluated during the course of the study. This data rep- 
resented each phase of the development process and ad- 





Projecl A 


Project B i 


1. Code size (KNCSS*} 


125 


77 


2. Hiimber of prerelease 
defects requiring fix 


269 


133 


3, Prerelease defect density 
(detects KNCSS) 


2.15 


t.ra 


4, Calendar months for 
prerelease QA 


9 


G 


5^ Total prerelease QA 
test hours 


3370 


2055 


6. Number ol postrelease 

defects repofled after one year 


2 


24 


7. Postrelease detect density 
(defects KNCSSj 


0.02 


0.31 


8w CaEendar months from 
investigation checkpoint 
to release 


23 


24 



dressed both quality and productivity issues [e.g., defect 
density and engineering hours). The results of the evalua- 
tion resulted in a set of recommendations that covered code 
inspections, development tools, testing, and people and 
process issues such as code reuse and code leveraging. 

Since every issue could not be addressed at once, we 
decided to find one area in the development process that 
had the greatest need for improvement and would provide 
the greatest return on our process improvement funds. We 
wanted to select methodologies and tools that could he 
used to improve the weak process area and couJd be easily 
integrated into our development environment. 

Process Improvement Area 

Fig. 2 shows the relative percentage of prerelease soft- 
ware defects based on the development phase where the 
defect was inserted. The data shown here is from Project 
8. but similar results were found for Project A. Initially we 
were surprised by this data. It might be assumed, for in- 
stance, that defects are Introduced into a product in equal 
amounts throughout each phase of development, or that 
the product design phase might be the most troublesome. 
However, the data presented here accurately represents the 
Waltham process, which of course may be different from 
other environments. 

Since 60% of the total prerelease defects were introduced 
into the product during the implementation phase, it was 
obvious that any improvement in this phase would yield 
the greatest benefit. During the implementation phase the 
activities that occur include coding, inspections, debug- 



*KNCS5: Thousands Of lines of noncomment source statements. 

Fig. 1 . Basic software quality metfics for two HP Waitham 
Division products 




-Design 28.2% (35) 



-Other 4.0% (5) 



Specification 2.4% (3) 
Optimization 2,4% (3) 
— Repair 1.6% (2) 



Implementation 61.3% {76} 



Fig. 2- Summary of defects by phase for proj^t B. 
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ging, and all tesUng^ 

Rnding a Methodology 

Closer iii%^estigation of our metrics data base revealed 
that some modules were more defect-prone than others. 
These troublesome modules consumed a great deal of time 
and effort during the implementation phase. Therefore, we 
needed a method to identify^ these modules early so that 
we could take the appropriate corrective measures^ such 
as more intensive code mspeclions. 

After examining the current software engineering litera- 
ture ^^^'^ and further scrutinizing of our project data, we 
found McCabe's cyciomatic complexity metric and its as- 
sociated methodologies best suited our needs. This metric 
provided us with a measure for detecting error- prone mod- 
ules and a methodology^ that fit right into our development 
process. The McCabe metric is a number that represents 
the complexity of a module. It is based on the number of 
decision statements in the module. It has been found that 
if the complexity measure of a module exceeds 10 the 
chance of that module being error-prone also increases. 
See the article on page 66 for more details- 

Fig, 3 shows some of the project data we used to help 
us evaluate the utility of the McCabe metric. This graph 
shows a comparison between prerelease defect density Had 
the complexity metric for programs belonging to project B 
(similar results were found for project A). Each program 
in Fig. 3 is a collection of many small modules and the 
complexity value shown is the sum of the complexity mea- 
sures for all of the modules in a particular program. From 
this data we were able to compute a 0.8 (or 64%) statistical 
correlation between complexity and defect density. 

Methodology and Tools 

The Mt:Cijhe mntric has been around for a while and its 
correlation between the metric and defect-prone modules 
has been validated in the literature/^^ "^^ We found the fol- 
lowing additional issues during our investigation of the 
McCabe metric and Us associated methodology. 
■ The algorithm for calculating the McCabe metric for each 

module is very simple and the process for gathering the 



data to compute the metric can be automated. 

■ The McCabe metric is expressed as a unit less number. 
Industrv' experience suggests that a complexit>^ measure 
in the range of 1 to 10 per code module is optimal for 
producing quality code. In fact, some organizations place 
a limit of 10 on all modules. 

■ The McCabe metric can play an important role in the 
module testiog process. A methodolog\^ has been de- 
veloped that allows determination of test paths and test 
cases using the complexity metric and the accompanying 
program flow graphs, 

■ The cyciomatic complexity of a code module can be 
presented graphically as well as numericallyT and there 
are tools for plotting representations of modules as cy- 
ciomatic flow graphs. 

Implementing Process Improvements 

The primary goal of this effort was to find a methodology 
that would help reduce the number of defects introduced 
into a product during the implementation phase of develop- 
ment. Once a methodology was found, our next goal was 
to integrate it into the real, heavily loaded, often skeptical 
R&D environment. We have successfully incorporated the 
McCabe methodology into our software development pro- 
cess by using it in early recognition of code quality* testing, 
code inspections, and the software quality engineering pro- 
cess. 

Recogition of Code Quality* As mentioned previously, the 
cyciomatic complexity of a module can be represented 
either numerically or graphically. As an example, consider 
Fig. 4. This diagram is the flow graph of a module written 
in C, which is part of a current development project at 
Waltham. This module has a cyciomatic complexity value 
of seven, which indicates a well-constructed module that 
may have a low defect density, or possibly no defects at 
all. The flow graph has been constructed using specific 
shapes to represent various programming structures. For 
instance, in this example IF statements are shown as 
branches and while statements are shown as loops. The 
complete syntax of a language such as C can be illustrated 
in this manner. The numbers on the flow graph correspond 
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Fig. 3, Comparison of defect ^en- 

sitieB and McCabe' s complexity 
for programs tn project B. The 
McCabe complexity value is the 
summatfor} of aft f^e complexities 
for the modutes m a partfcular pro- 
gram. Reused code is code that 
fs used witf] no changes, and lever- 
aged code is modified reused 
code. 
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The Cyclomatic Complexity Metric 



The quaniiticaiion of program complexity is related to the 
number of decisions (changes in control) in the program, This 
is opposed to the viewpoint that complexity can be quantified 
from program size or the number of independent program paths. 
Program si^e is misleading becaLfse a large program may have 
very few decision statements For example, a 2-KNCSS program 
may have only one or two decfsions implying one or two paths, 
whereas a 50-Jine program with 25 if-then statements in sequence 
coutd generate 33.5 million paths, Basing code complexrty strictly 
on Ihe number of paths is also misleading because the number 
of paths can be infinite for programs that have loops. 

To provide a metric that indicates a meaningful set of program 
paths, the cyclomatic complexity metric quantifies a basic 
number of paths that have the following properties: 

■ They visit every node (program statement) in a graph of the 
program, and they visit every e6ge (change of control) m the 
graph 

■ When taken together the basic paths can generate all possible 
paths in the program, 

To develop these concepts, a definition and theorem are 
needed from graph theory 

Definition 1 The cyclomatic number v(G) of a graph G with n 
nodes, e edges, and 1 connected component is; 



y(G) 



n + 1 



A connected component Is a code module (function or proce- 
dure) from start to end. 



Nodes 



m. 




Edges 



m 



(b) 




Fig. 1 . a) Program flow graph for a program with severe nodes 
(btocks of code} and ten edges (branches), b) Same control 
graph with added edge to satisfy the reguirement that the 
graph must be strongly connected 



b^ 


a beg 


b2 


a(bc)-2g 


b3 


abefg 


b4 


adefg 


bs 


adtg 



Theorem 1. Jn a strongly connected graph G, the cyclomatic 
number is equal to the maximum number of llnearfy independent 
paths. 

To apply this theory a program must be represented as a 
directed graph in which a node represents a sequential biock 
of code, and an edge corresponds to a branch (transfer of con- 
trol) between nodes (see Fig 1) it is assumed that each node 
IS entered at the beginning and e>{its only at the end. 

The program flow graph in Fig 1 has seven blocks (a through 
g). entry and exit nodes a and g. and ten edges To apply the 
theorem the graph must be strongiy connected, which means 
that, given the two nodes a and b, (here exists a path from a to 
b and a path from b to a To satisfy this, we associate an additional 
edge with the graph that branches from the exit node g to the 
entry node a as shown in Fig, lb. Theorem 1 now applies, and 
it states that the maximal number of states in G' IS 11 ~ 7 -F 1 - 5. 
The implication is that there is a basic set of five independent 
paths that when taken in combination will generate all paths. The 
five sets of paths for G' are: 



' (bc)'2 means iterate loop be twice } 



ff any arbitrary path is chosen, it should be equal to a linear 
combination of the basis paths b1 lo b5, For example, the path 
abcbefg is equal to b2 + b3 - bl , and path a(bc)*3g equals 
2-b2 - bt 

The general form of the complexity metric for a module is 
v(G) = e - n -I- 2, The association of an additional edge from 
exit to entry for each module is implicit. Therefore, we have; 

v(G) = (e + 1)-n-h1 ==e-n-h2 

Applications of v(G) 

The cyclomatic complexity metric has applications in the fol- 
lowing areas: 

■ The cyclomatic number has often been used in limiting the 
complexity of modules in a software project. Experience and 
empirical data^ have suggested that there is a step function 
in defect density above a complexity of 10. Therefore, U is 
good practice to limit the complexity of each software module 
to 10. The one exception to this rule is a case statement, which 
contains an arbitrary number of independent paths. This struc- 
ture can have a high cyclomatic complexity and have a low 
defect density 

■ The cyclomatic number can be used as a predictor of defects. 
Various projects have correlated cyclomatic complexity and 
defect density and have reported correlations of 0.8 as in the 
accompanying paper to 9.6.^ 

■ The cyclomatic number and the accompanying program con- 
trol flow graph can be used to identify test cases. The cyclomat- 
ic number corresponds to the number of test paths and the 
test paths correspond to the basic paths derived from, ihe 
control flow graph. With this information, test conditions (test 
cases) can be generated for a program, This entire process 
can be automated with a language dependent preprocessor 
The following example illustrates the derivation of test paths 
and test cases using the cyclomatic number and the control 
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graph 

The foKowiBg procedure in C solves the weii-known triangle 
grapti pfoblem. whsch lakes three integers and determines what 
type of !rj angle they reprsserrt 



Node numbers 




in Rg 2 









main 


{ 


1 




int a. 


1 




b. 


1 




c, 


2 




scant ("%d%d%d" a^a.&b.&G); 


3 




if (a > = b + c) 


4 




pnntf ("no triangleNn"); 


7 




else 


7 




[f (b > = a + c) 


8 




pnntf ( no iriangle\n"): 


10 




else 


10 




if (c > - a + b) 


11 




printf ("no lnangle\n"): 
else 


13 




13 




If (a = b) 


14 




if (b - c) 


15 




printf {"eq ui fateraiNn" ) ; 


18 




else 


16.18 




printfC'1sosce(es\n"); 


19 




else 


19 




if [a = c) 


20 




printf (' ' isoscel es\n " ) ; 


22 




else 


22 




if(b =c) 


23 




p rin tf ( " i soscelesVi ' ' } 


25 




else 


25.24,21 


J 7,12,9,6.5 printf ("scaleneMi"); 


5,6 } 







This program has 32 edges and 26 nodes giving a complexity 
of eighit (see Fig 2) This means there are eight test paths for 

thiis program. The following list shows the first five test paths and 
the test conditions for each path. 



Test Path 

1 0123456 

2 12378956 



Node 



Test Conditions 



3 (a ^-h + c) -* TRUE 

3 (a >= h + c) -^ FALSE 
7 fb >= a + G) — TRUE 




^ O © ^^ 



r 


^^ <» w - 


Fig. 2. Program flow graph for the triangle graph probie 


01 2371011 12956 
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(a >= b + c) "t FALSE 
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(b >= a + c) -* FALSE 




10 


(C >-a + b) -» TRUE 


01 23 7 1013 192021 17 


3 


(a >= b + c) -* FALSE 


12956 


7 


(b >= a + c) -» FALSE 




10 


(c >= a + b) -^ FALSE 




13 


(a = b) -* FALSE 




19 


(a ^ c) -* TRUE 


1 23 7 10 13 14 18 16 17 


3 


(a >* b + e) "» FALSE 


12956 


7 


(b >= a + c) -^ FALSE 




10 


(C >= a + b) -- FALSE 




13 


(a = b) -* TRUE 




19 


{a ^ Cj -»- FALSE 



The syiTibol -* indicates that the conc^itiona in parfiniheajs must be set 
to TRUE or FALSE 
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to actual C source statements. This representation provides 
a useful reference between the flow graph and tJie code. 

Fig. 5 is a similar diagram for another C code module 
from the same development projet:t= Note here that the 
cycloniatic: f;omplexity is 36, and that the flow graph for 
the code is more complex. Since this module's complexity 
metric exceeds the optimal vahte of 10 it is hkely that this 
module will be error-prone. In addition, Pig. 5 provides 
visual evidence that a complex module may be hard to 
understand, test^ and maintaio. 

Our experience at Waltham indicates that the graphical 
representation of code complexity is a very effective vehicle 
for focusing lab-wide attention on code quality. The visual 



impac! of an image of tangled code appears to attract more 
interest than mere correlation of numbers. Therefore, cur- 
rent projects are actively using cyclomatic flow graphs dur- 
ing the coding process to focus engineering and manage- 
ment attention on code quality. 

Testing Process, The test case generation capability of the 
McCabe methodology has been very useful in establishing 
rigorous modulo testing procedures. The cyclomatic com- 
plexity valuRS have been used as an indicator of which 
modules should be subjected to the most active scrutiny 
by the test group. Modules with abnormally high complex- 
ity values are selected as candidates for the most extensive 
test activities. 
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Cyclomatic 36 



CyclDmatlc 7 



hiodes 



Upward 




Fig, 4, The progra m flow graph for a modu!e with a cyctomalic 
complexity of 7. 

Code Inspections. Recent studies have suggested that one 
of the most effective techniques for software defect preven- 
tion and detection is the use of formal inspections/^ The 
complexity data and the flow graphs can be used to help 
evaluate various code paths during an inspection, and to 
help determine which modules should be given an inspec- 
tion. 

The Software Quality [engineering Process. The software 
quality engineering (SQK) group at Waltham has been ac- 
tively promoting the use of McCabe's technology within 
the lab. Specifically, the SQE group is working with current 
projects so that all code is subjected to calculation of the 
cyclomatic complexity of each module. This process has 
been established as part of our software product release 
criteria. In addition, the SQE group has purchased and 
maintains a tool* that computes complexity and generates 
program flow graphs. As each project completes major 
blocks of code, the SQE group generates the flow graphs 
far that code and then provides feedback to project manage- 
ment and team members. 

Conclusion 

The McCabe methodology and toolset have been inte- 
grated into the Waltham software development process 
over the past year. This process has been accomplished 
with no disruption to current lab projects and has resulted 
in the following successes: 




F[g. 5. The program flow graph for a modufe with a cydomatlc 

compiexity of 36. The high comptexity vatue ar}d the visual 
presantation indicates that this module is error- prone and 
very tikety hard to mairytain. 

■ Automatic identification of potentially faulty software 
before actual testing is started 

■ An tomat i c i d enti f i cati o n of co de mo d ules that could ben- 
efit from code inspections 

■ Automatic generation of test case data for all software 
modules 

■ Well-defined coding standards accepted throughout the 
lab 

■ Effective code defect prevention strategies based on re- 
structuring of overly complex code. 

Each of these successes has contributed to the overall 
success of the software defect prevention program pres- 
ently underway at the Waldiam lab. By identifying and 
correcting software code defects very early in the coding 
phase of product development, the McCabe methodology 
and toolset continue to have a major impact on our efforts 
to improve the productivity of the Waltham development 
process ajid the quality of the resultant products. 



•ACTw (ArnaJysis Of Cofnf^s% Tool) a pfoducE of McCabe and Associate 
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Object-Oriented Unit Testing 

HP's Walt ham Division has taken a first step in applying new 
and traditional unit testing concepts to a software product 
Impiemented in an object-oriented language. 

by Steven P. Fiedler 



ALTHOUGH OBJECT-ORIENTED ENVIRONMENTS 
are being used more frequently In software develop- 
ment» little has been published that addresses ob- 
ject-oriented testing. This article describes the processes 
and experiences of doing unit testing on module.? devel- 
oped with an object-oriented language. The language is 
C+ + ^ and the modules are for a clinical information sys- 
tem. Because the system must acquire real-time data from 
other devices over a bedside local area network and the 
user requires Instant information access, extensions were 
made to the language to include exception handling and 
process concurrency. We call this enhanced version Ex- 
tended Ch- -h. Test routines were developed and executed 
in an environment similar to that used in development of 
the product. This consists of an HP 9000 Series 300 HP-UX 
6.01 system. 

Unit Testing 

Unit testing is the first formal test activity performed in 
the software life cycle and it occurs during the implemen- 
tation phase after each software unit is finished. A .software 
unit can be one module, a group of modules, or a subsystem, 
and depending on the architecture of the system, it is gen- 
erally part of a larger system. Unit tests are typically de- 
signed to tesl software units, and they form the foundation 
upon which the system tests are built. Since software units 
and unit tests are fundamental entities, unit testing is crit- 
icalto ensuring the finiil quality of the completed system. 

The unit testing prof:ess involves test design, construc- 
tion, and execution. The test design activity results in a 
test plan. Because the primary intent of unit testing is to 
find discrepancies between unit specifications and the 
coded implementation,^ the unit specification is the pri- 
mary reference for the test plan. Test construction involves 



building the test cases based on the test plan, and test 
execution involves performing the tests and evaluating the 
results. 

Both structural [white box) testing and hinctional (black 
box] testing techniques are used in unit testing. Since struc- 
tural testing requires intimate knowledge of the design and 
construction of the software, unit testing requires intense 
developer involvement in the process. 
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Fig, 1. (a) Object instance objn being sent the message 
rnove item fxnew.y new) 10 invoke the method to move a graphi- 

cal ftem from one focation to another, (bj The same operation 
being processed in a procedural language environment 
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Objects 

An object is the fundamental building block in an object- 
oriented environment and it is used to model some entity 
in an application. For example, in an office automation 
system, objects might include mail messages, documents, 
and spreadsheets. An object JS composed of data and 
methods. The data constitutes the information in ihe object, 
and the methods, which are linalogous to procedures and 
functions in non-object-oriented languages, manipulate the 
data. In most applications, there are many objects of the 
same kind or class (e.g.. many mail mes.'>ages, devices, etc.). 
C+ + def in es the data and met hods for these s i mi lar objects 
in a data type called a class. Each object in an object- 
oriented language is an instance of a particular class. Also 
in C+ + , a data item is referred to as a member and the 
methods, member funcljons. 

One of the main differences between object-oriented and 
procedural languages (non-object-oriented languages) is in 
the handling of data, in a procedural language environment 
such as Pascal C. or Fortran, system design is based on 
the data structures in the system, and operations are per- 
formed on data passed to procedures and functions. The 
primary data items are typically global and accessible to 
all the modules in the system. In an object-oriented envi- 
ronment, the object's internal data structures and current 
values are accessible only to the methods within the object. 
The methods within an object are activated through mes- 
sages passed from other objects. The messages indicate the 
method to be activated and any parameters required, 

Fig. 1 illustrates these diferences in architecture between 
uhject'Oriented systems and prot:edural-langijage-ba.sed 
systems. In Fig, Ta, to move a graphical item (objx) from 
one location to another, the message move_item(xnew,ynew} 
is sent to the object instance objx to perform the operation. 
The current location and geometric characteristics of the 
item are contained in the data structures of objx. The 
methods in objx will handle the transformation and transla- 
tion of the item to a new location. Fig. lb depicts the same 
operation in a procedural language environment. The 
graphical items 's data structure and current values are kept 
in a global data structure which is accessible to all the 
modules in the system. 

Objects and Unit Testing 

rhe issues related to objects and unit testing Include: 

■ When should testing begin? In a procedural language 
environment, a complete unit may not exist until several 
functions or procedures are implemented. In an object- 
oriented environment, once a class has been defined and 
coded, it can be considered a complete unit and ready 
for use by other modules in the system. This means that 
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unit testing must be considered much earlier in an object- 
oriented environment. 

■ What testing techniques should be used? Since the 
paradigm of obj eel-oriented programming emphasizes 
the external behavior of data abstractions rather than the 
internals, one would expect to employ only black box, 
functional testing techniques. However, a more robust 
testing structure emplojring complete path testing is ac- 
tually needed. 

■ What should be tested? In an ideal situation, the answer 
to this question would be that all classes should be com- 
pletely path tested, particularly for critical application 
systems. However, the resources required to meet this 
goal may be substantial and, in working towards it, trade- 
offs are likely to be made. Nonetheless, refinements can 
be added to the testing process that simplif>^ labor inten- 
sive phases and improve chances that a minimal set of 
tests will be executed. 

m Who should do unit testing? To answer this question, 
we need to consider what is being tested and the exper- 
tise required of the tester. Remember that units are typ- 
ically modules that eventually become part of a larger 
system and only the developers know the detailed inter- 
nals of the units they are responsible for building. As a 
result f an independent tester or a developer who is not 
involved in the design and generation of code for a spe- 
cific class may find it difficult to perform adequate test- 
ing on that class. For example, a developer may design 
a data base class w^hich is intended to make it easier for 
a user to perform transactions in a data base. The methods 
within the data base class are responsible for performing 
the data base interface tasks. An independent tester who 
is unfamiliar with the way in which these low-level func- 
tions work would certainly be ineffective in testing the 
internals of this class. 

In the clinical information system, knowledge of Ex- 
tended C+ -I- was sufficient to become an effective tester 
for certain classes in the system. This was because of the 
formulation of generic classes, A generic class in the clin- 
ical information system is a class that provides general 
functionality. It can be considered an extension of the lan- 
guage's built-in data types that fills a utiliti^rian purpose 
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1. String contains no characiers- 
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for otber components of tbe system. Strings and linked 
lists are examples of objects that provide such universal 
functionality. 

To build on this generic concept, parameterized type 
classes were introduced.^ Farameterization permits a gen- 
eral definition of a class to be extended to create a family 
of type-safe classes, ail with the same abstract behavior. 
For example, suppose we design a class called Array which 
contains pointers to some object- Through parameteriza- 
tiQn» we can extend this class definition to create arrays 
that point to characters, arrays that point to strings, or 
arrays that point to any other type of object (Fig, 2]. The 
testing of a parameleri/,ed type class can provide a high 
level of reiiabihty for a growing family of similar classes. 
From the experience gained in testing generic classes, we 
have developed an approach to the testing of other C-^ + 
classes. 

Test Process 

The tasks associated with the testing process for objects 
are the same as for regidar unit testing: design, construction, 
and test execution. 

Design, During the design phase, the tester determines the 
test approach, what needs and does not need to be tested, 
the test cases, and the required test resources. The inputs 
required to conduct the design phase for objects include: 

■ The header and source files of the target class [the class 
being tested), and a well-defined specification of the 
ciass."^ An example of a class specification is shown in 
Fig^ 3. 

■ An analysis of the effects of inheritance on the target 
class. When a class uses another class as a base to build 
additional functionality, it is said to be derived from 
that class and consequently iulierits data and methods 
from the base (parent] class. If the target class is derived, 
we want to know if the base class has been thoroughly 
tested. Provided that the functionality of the base class 
has been proven, any member function of the large! test 
class that leverages directly from a base class member 
function will require minimal testing. For example, the 
specification in Fig. 3 shows that String is derived from 
a parameterized class called Sequence. The functions that 
Siring inherits from Sequence (AddRrst, Capacity, etc, J require 




Fig, 4, Path lest cases far the function Lowercase. 



Fig. 5. Dependencies for class String. 
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only a basic functionality test- 
m The cycJomatic complexity metric'' of the individual 
member functions belonging to the target class. The com- 
plexity measure and its accompanying testing methodol- 
ogy play a key role in the implementation of this test 
strategy. Through their use, we can ensure that all the 
independent paths in the target member functions are 
tested. Fig. 4 shows an example of path test cases for 
the member function Lowercase, hi our project, the predi- 
cate method^ of calculating cyclomatic complexity has 
been built into the Extended C + + parser: 

■ A hierarchy or structure list which shows member func- 
tion dependencies. In simple terms, what member func- 
tions call what other member functions of this class? 
Private member functions, which are not accessible to 
the end user directly, should also be included in this 
list. For example, in Fig. 5, the function operator + 
performs its task by invoking the Append function, which 
indicates that Append should be tested first, 

■ The signals or exceptions that are raised (not propagated] 
by each function, Extended C-h + includes linguistic 
support of exception handling/ which permits a special 
kind of transfer of control lor processing unusual but 
not necessarily erroneous conditions. These signals 
should not be confused with HP-UX operating system 
signals. Signals are defined for the various member func- 
tions in a class specification, For example, the specifica- 
tion for String indicates that the member function OeJete- 
String raises a signal called InvalidStartlncfex if the Startlndex 
parameter passed to the member function is not valid. 
The last step of the design phase is to dHtermine tlie 

approach to use to verify the test results. There are a number 
of options in this area. One approach is to print out the 
expected results for a test case and the actual results gen- 
erated by the target class test to two different files. At the 
end of the test, the two files can be compared using the 
standard UNIX- tool diff [see Fig. B), A second option for 
results verification uses similar ideas, but may require less 
actual programming time. An expected results file can he 
constructed by hand and the file can be used for comparison 
with actual target class output, if these two approaches 
prove impractical because of the behavior of the class being 
tested, a third alternative might be to include the expected 

UNIX IS a fegi^sred TrademarK of AT&T in the U.S A. and other gguntnes 
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observations in a test plan using the class specification as 
a basis for deriving these observations. 

Fig. 7 shows an excerpt from the test plan for the class 
String, A test plan is the culmination of the test design 
process, and in addition to guiding test activities, it is an 
excellent respository of information regarding what was 
done to test an object. 

Construction and Execution, The strategy for developing 
test cases for execution is to determine all the paths in a 
module that require test coverage, and then to create test 
cases based on the class specification [black box approach) 
and certain features in the code (w^hite box approach). The 
white box strategy is based on the structured testing 
methodology^ resulting from McCabe's work (see article on 
page 64 for a discussion of the use of the McCabe complex- 
ity metric in our division). In this methodology, test cases are 
created to execute each decision path in the code. In the 
clinical information system, except for paths that contained 
code for exception handling, test cases were written to 
ensure complete path coverage of each member function. 
Exception handling situations w^ere dealt with separately 
because they disrupt the normal control flow of a program. 
Based on the class specification and source code, test cases 
designed to ensure path coverage were derived using the 
other well-understood methodologies of equivalence parti- 
tioning and boundary-value analysis.^ 

In creating test cases for valid equivalence classes, realis- 
tic input values for the member functions were preferred 
over those that lacked relevance from an application 
standpoint. For example, if the primary use for our sample 
String class is to hold a single line of information on an 
electronic index card, w^e might expect it to hold, on aver- 
age, 12 to 50 characters. Our test case would be to create 
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a string of 40 characters rather tlian 140, 

Boundary- value analysis dictates that tests be built that 
create objects of extremes. Instances of null strings 
(si^e = 0) should respond as any non-oull string would 
unless the specification states otherwise. Clearly, a null 
string appended with the value a^c should yield the same 
result as the string abc appended with a null value. At the 
other extreme, tests should exist to stress objects (usually 
in size] beyond al) expectations of normal use. For example, 
in HP-UX. main memory is managed in pages of 409 B bv^es. 
Therefore, it should be valid to create a string that holds 
4097 characters. 

Tests to invoke exception handling capabilities were also 
included in class test suites* Boundary-value conditions 
were used to invoke these facilities. For example, if an 
exception is encountered when we index beyond the legal 
boundary of a siring, the test case invokes the exception 
by trying to access the character just past the end of the 
string, not ninety-nine characters past it. Special care must 
be taken in coding exception test cases, because if a signal 
raised by a member function is not handled correctly, an 
aborted test program may result. 

There are other areas for test cases that do not show up 
using the structured technique. For example, the effects of 
implicitly and explicitly invoking a class's constructor and 
destructor^ functions should be examined for consistencyH 
Inilialli^ation and casting operations should also he tested. 
In addition, delects have been discovered by applying as- 
sociativity rules to member functions. That is, if string si 
is null, and string s2 is not null, si > s2 should yield the 
same results as s2 < s1. In addition, the use of the object 
itself as a member function input parameter proved valu- 
able in uncovering subtle implementation errors. For in- 
stance, given s1 is a string, the test si Append(sl) becomes a 
legitimate and creative way of triggering certain test condi- 
tions. Much of this type of testing can be integrated into 
standard testing without creation of separate tests. 

Results 

The methodology presented here was applied to testing 
several generic classes after the development group had 
completed their testing using black box testing techniques. 
The results show the shortcomings of strict black box test- 
ing. Even though development group testing was extensive 
and appeared to be thorough* defects were still uncovered. 

^fri C + + a consUuctOf and a destrucfdr parlo-nm ir>e rniiialnaiion and termfnotJCin Igr cl^ss 



Defects were found in each of the generic classes tested. 
The number of defects found seemed to be related to the 
composite (total} complexity- of all of the class member 
functions and more directly to the number of noncomment 
source statements [NCSS) contained in the source and in- 
clude files. The general relationship of complexity to de- 
fects is shown in Fig. Sa. and the correhttion between de- 
fects and the NCSS of each class is shown in Fig. 6b. Each 
point represents a generic class. On average, a defect was 
uncovered for every 150 lines of code, and correspondingly, 
the mean defect density exceeded 5.1 per 1000 lines. Only 
the code contained in the source and include files for each 
class was counted for this metric. Code from inherited func- 
tions was not considered. These defect rates pertain to a 
small set of actual product code produced during the early 
stages of development. Another interesting relationship 
was obser\'ed when the NCSS values of source and test 
code were compared (see Fig. 9J- 

Conclusion 

Tiiere is a cost associated with class testing. A significant 
investment of time is required to perform the testing pro- 
po.sed here. Assuming testers are already competent with 
the object-oriented environment » they must acquire famil- 
iarity with McCabe's complexity concepts as well as a basic 
understanding of the class being tested- Because testing so 
far has taken place concurrently with development, time 
estimates for the testing phase have been somewhat incon- 
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sisteii! and do not yet suggest any clear concliisions* Fig. 
10 summarizes the metrics we have coJlected thus far, (The 
classes are listed in the order they were lested). 

In the uhject-oriented eriviroiiniBtit, objects and their def- 
initions, rather than procedures, occupy the lowest level 
uf progtcim Hpeci float ion. Therefore, it is necessary to focus 
on them when implenieiiting a thorough test methodology. 
Practices used in testing tradilional procedural systems can 
be integrated in the approach to object-oriented testing. 
The maiti difference we have found so far is that each object 
must be treated as a unit, which means that unit testing i[i 
an ubject-oriented environment must begin earlier in the 
life cycle. Through continued collection of the class metrics 
and test results, we hope to gain more insight and continue 
to improve our object-oriented unit test efforts. 
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Validation and Further Application of 
Software Reliability Growth Models 

After two years of use, a software retiabillty growtt) model 
has been validated with empihcaldata, and now it Is being 
expanded to estimate test duration before it begins. 

by Gregory A. Kryger 



AT HP'S LAKE STEVENS INSTRUMENT DIVISION, 
a software reliability growth model* has demon- 
strated its applicability to projects ranging in size 
from 6 KNCSS to 150 KNCSS [thousand lines of noncom^ 
ment source statements] , and in function from instriiment 
firmware to application software. Reliability modeling 
curves have been used to estimate the duration of system 
integration testing, to contribute to the release-to-sales de- 
cision, and to estimate field reliability. Leveraging from 
the basic model, project managers are beginning to plan 
staffing adjustments as the QA effort** moves through the 
defect-fixing-limited phase and into the defect-finding-lim- 
ited phase. 

Basic Model 

III \Uii iiiW of 1986, a software reliafollily growth model's 
good fit to historical data on a previous firmw-are product 
led to Lhe development of a set of release criteria, with 
defects per system test hour (QA hour) as the principal 
quality measuren^ The model and release criteria were then 
applied in real time to a new application product. The 
modeling effort aided in predicting when the product was 
ready for release to customer shipments and provided es- 
timates for the number of defects thai might be foimd in 
the field* 



'So^lvi^are reEiab'-lily gfowtti modeling ■-£ bas^id cin lhe premtse that as so^are is lesTed 

ar>d defeeia removed, the mliabH^tv gels better {gtows) 

"QA flrtoil CJF QA phase In Ibis paper meters to lhe system rnt^ratton Cest phme of the 

soltwafB- ^lte cycle 



The basic exponential model is based upon the theory 
that lhe software defect detecUon and removal effort will 
follow a nonhomogeneous Poisson process. "^ Li this process 
the defect arrival rate is assumed to decrease with every 
hour of testing (or at least with every code correction]. The 
model has two components. 

The cumulative number of defects found by time t is 
given by 

m(t] = a[l-e]-^*^''*^^ 

and the instantaneous new defect-finding rate at time t is 
given by 

l(t) - ke-*'^''*^^ 

Fitting the model requires the e.stimation of parameters 
k, the initial defect discovery rate, and a, the total number 
of defects. The data required is obtained by recording on 
a daily or weekly basis the time spent executing the soft- 
ware and the resulting number of defects discovered. The 
model parameters may be estimated by the least squares, 
nonlinear least squares, or maximum likelihood methods 
In most cases, the maximum likelihood method is pre- 
ferred . 

Considering typical software development and system 
testing practices* the assumptions necessary for the 
applit:ability of Poisson theory would seem to negate the 
use of the model. Key assumptions of the tnodel and the 
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correspondingly realities are; 

■ Assumption: All functionality is completed before the 
start of system testing. 

Reality: Many products enter system testing without all 
the feature;? in place. 

■ Assumption: Testing can be considered to be repeated 
random samples from the entire input domain. 
Reality; There is some random testing, but typically test- 
ers are more structured and systematic in the selection 
of test cases. 

M Assumption: Defects found are removed with certainty 
and no new defects are introduced fa perfect repair). 
Reality: A defect repair may introduce new defects. 

■ Assumption: The times between failures are indepen- 
dent. 

Reality. When a defect is found in a particular area of 
the software, because of the suspicion that there may be 
more defects in the same area, the area is probed far 
more defects. This process usually finds more defects, 
which is good, but makes the arri%^3l rate of defects de- 
pendent on when the last one was found. 
As has been said, with such a set of assumptions, it 
would seem unlikely that this model would fit real- world 
data, However, some aspects of the testing proLiess at Lake 
Stevens approximate these conditions. First, our life cycle 
calls for all functionality Id be completed by the time we 
start formal system integration testing. Typical pro) eels 
have 95% or more of their functionality complete by this 
time. Second, the entire set of functionality is subdivided 
and assigned to different individuals of the testing team. 
Therefore, while the testing process cannot be considered 
to be repeated random samples from the input domain, it 
is at least sampling from the entire functionality set as time 
progresses. This is in contrast to a testing process wherein 
some subset of the functionality is vigorously tested to the 
exclusion of all others before moving on to another subset 
and so on. Regarding the third assumption, strict revision 
control procedures at least maintain some control over the 
rate of defect introduction. Finally, nothing about the Lake 
Stevens development process justifies the assumption that 
the times between failures are independent. After finding 
a serious defect in a ptirtion of the product ^ testing effort 
often intensiBes in that area, thus shortening the next time 



to failure. 

The model's success in describing the projects at LSID 
demonstrates some degree of robustness to these assump- 
tions. Our past and continued application of software relia- 
bility theory is not based on a fundamental belief in the 
validity of the assumptions, but in the empirical validation 
of the model. Therefore, we have continued to use software 
reliability growth models with the following objectives in 
mind: 

■ To standardize the application of the model to all soft- 
ware products produced at LSID 

■ To put in place a set of tools to capture and manage the 
data and obtain the best fit curves 

■ To use the defect-finding rate and the estimated defect 
density to define the release goal 

■ To pred i ct the duration of the QA phase before its start 

■ To understand the relationship between model estimates 
and field results. 

Standardized Application 

To date^ .software reliability growth modeling has been 
conducted on eleven projects thai have since been released 
for customer shipment. Two demonstrated excellent fit to 
the model, two very good fit, four showed a fair confor- 
mance to the model, and three showed a poor fit. Fig, 1 
shows the curves for one of the projects on which the model 
gave an excellent fit. Contrast these results to the modeLs 
performance on the project shown in Fig. 2. Note that time 
in this case is measured in calendar days rather than test 
hours. Here the cumulative defects begin to taper off only 
to start up again. These results reflect inconsistent testing 
effort, which is not picked up by simply measuring calen- 
dar days of testing effort. The curves in Fig. 2 were obtained 
by independently fitting the basic model before and after 
the change in testing effort. These two best-fit models were 
then tied together to form the piecewise curves shown. 

Tools 

The defect tracking system (DTS),' an internal defect 
tracking tool, is used by all project teams to log defects 
found during system testing. In software reliability modeling 
it is important to record all time spent exercising the soft- 
ware under test regardless of whether a defect is discovered. 
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DTS has proveD to be unsatisfactory- for capturing QA hours 
that do not produce a sofUvare defect. Therefore, project 
teams separately log test hours at the eod of each day. 

The DTS data is loaded into an Informix data base so 
that it can be sorted and retrieved as desired. On projects 
using DTS for tracking QA time as well as defect statistics, 
Iiiformix reports generate files with weekly (or daily) QA 
hour and defect total data pairs. On projects tracking QA 
time separately, the weekly (or daily! defec:l totals are re- 
trieved from the Informix data base and matched with the 
appropriate QA hours. In either case, the file of cumulative 
QA hours and cumulative defects found b submitted to a 
program that obtains the best-fit model parameters by the 
method of maximum likelihood. At the present time, plots 
for distribution are generated using Lotus* 1-2-3*'. Future 
plans call for using S. a statistical package that runs in the 
HP-UX environment, to generate I he graphics, thereby con- 
ducting the data manipulation, analysis, and plotting ail 
on one system. 

Reiease Goal 

The software modeling process provides two related met- 
rics that help support a release-to-customer'Shipments de- 
cision: the defect-finding rate and the estimated number 
of unfonnd defects, A specific goal for one of these two 
metrics must be established if the model Is to be used for 
predicting the conclusion of system testing. 

The defect-finding rate is 3 statistic you can touch and 
feel. It can be validated empirically — for example, 100 
hours of test revealed four defects. On the other hand, one 
can never really measure the number of defects remaining. 
This mefrit: can only be estimated. Although the two mea- 
sures are related, it Is riot true that two projects releasing 
at the same defect-finding rate goal will have the .same 
number of defects estimated to be remaining. Couple this 
fact with the recognitiuo thai the sizK tif the product has 
no bearing on the mociel lit and the resulting estimated 
number of residual defects and it is clear thai two projects 
releasing at the same find rate could have quite different 
estimated residual defect den.sities. Because of its observa- 
bility, the defect-finding rate has been used as the principal 
release goal on all projects In date except one. However, 

LQ1U5 and 1-;!'3 are US regialeted tradfimarks of Lotus CtevaEopment Corporal ion 
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both the failure rate and the estimated residual defect den- 
sity are monitored and used in aiding the release decision. 

The Project E Experience 

The one project to date using a goal of ending system 
test wllh a certain residual defect density will serve as a 
good n lust rat ion of the contributions and limitations of 
software reliability growth models. Project E is an applica- 
tion softw^are product of 156 KNCSS. This project repre- 
sents a new release of a previously developed product and 
is roughly tw^o-t birds reused or leveraged rode. The stated 
goal at tlie start of system integration testing was to achieve 
an estimated residual defect density of 0.37 defects per 
KNCSS. a goal derived from the performance of the first 
release of ibis product. Such a goal means that the bcst-fit 
model should he eEitinialing f}3 residual defects. 

A team of engineers was assembled to conduct testing 
while the project team fixed defects. The data was plotted 
at roughly 30-hour testing intervals and the model refit 
each week, Thn most recent curve was used to estimate 
the QA hours required to achieve the objective and these 
estimates werti plotted weekly with statistical confidence 
limits as shown in Fig. 3. In mid-April, Ihe decision was 
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made to release the projecl for customer shipments and to 
continue futher testing and refinements for a final release 
in June. Tfis team had all but reached the goal and the data 
had tracked the model very vvf^ll. At this point, the en- 
gineers on the testing team disbanded and returned to their 
original project assignments. The design team then took 
on the task of conduntinjj both ccmtinued testing and defect 
resolution. With only the desij^ners looking, the defect dis- 
covery rate jumped up rather than continuing to follow the 
curve as can be seen in Fig, 4. The designers were testing 
specfic areas of the code [directed testing), so an hour of 
testing now was not equivalent in intensity to an ho or of 
testing with the previous tftsl team. The testing process 
was not meeting tlie assumption that testing can be consid- 
ered to he repeated random samples from the entire user 
input domain. 

What is clear from tliis project is that the failure rate data 
and curves are tnodeling more than the software product 
alone. They are modeling the entire process of testing. The 
estimates of failure rates and residual defect densities are 
estimates only as good as the testing process itself. The 
degree to which these s^tatistics match field results will 
depend upon the degree to which the testing matches the 
ciL-^tomer's use profile. The identification of the customer's 
use profile and the incorporation of that information into 
the testing strategy is a topic for further investigation. 

Before QA Begins 

Naturally wo would like to estimate the duration of the 
QA phase before it begins. But fitting a model to do estima- 
tion must wbU for testing to begin and for enough data to 
be collected before an effective statistical analysis can be 
conducted. However^ it is possible to use results from past 
projects to estimate the two model [jarameters a and k. 

In preparation for testing a recent softw^are product, Proj- 
ect F, we reviewed the total number of defects discovered 
during system integration testinf^ on past projects. Defect 
densities appeared to fall between 12 and 20 defects per 
KNCSS. Project F had 28.5 KNCSS. so the likely range for 
tlie first model parameter, a, was calculated to be 342 In 
570 defects. Again looking at past projects, the initial defect 
discovery rate averaged around one defect per hour^ so the 
other model parameter, k. could be set to one. Given a goal 



for the failure rate of 0.08 defects per hour, an expected 
range of 864 to 1440 QA hours was calculated. 

Management ultimately needs an estimated date of com- 
pletion so the expected QA hours required for sy.stera test- 
ing must be converted to calendar time. To accomplish this 
we again reviewed the data on past projects and discovered 
an amazing consistency of four QA hours per day per per- 
son doing full-time testing, and an average of 2,3 defects 
fixed per day per person doing full-time fixing. Given the 
number of team members capable of fixing, the number 
capable of finding and those qualified to do both, the re- 
quired QA hours for testing could now i^e converted to 
calendar time. Fig> 5 show^s the final QA projections for 
Project F and the staffing levels used to convert the QA 
hours into calendar time. Note that the staffing levels given 
correspond to the midrange assumption of Ifi defects per 
KNCSS. 

Recognize that as testing proceeds, testing and fixing 
resources will have to be shifted. Early in the process, the 
project is fixing-constrained because a few^ testers can find 
enough defects to keep all available fixers busy- Overtime 
this changes » until late in testing, the project is finding-con- 
strained since it takes many resources looking for defects 
to keep only a few fixers working. Also, the finders cannot 
be allowed to outstrip the fixers, creating a large backlog 
of unresolved defects. Such a situation only causes frustra- 
tion for the finders Ijecause of testing roadblocks created 
by defects already found. 

Our experience to date with using the model to estimate 
the duration of the QA phase before its start demonstrates 
the difficulty in estimating the two required model param- 
eters without actual system test data. Project F concluded 
system integration testing with a total QA effort that was 
225% of the original effort, Over twice the expected number 
of defects were found and resolved, Not all of this error in 
estimation can be blamed on the failure of the motlel. In 
hindsight, w^c completely disregarded the fact that this was 
not a stand-alone project. Many of the problems encoun- 
tered vvere because Pro jet: t F had to he integrnted with 
another 1 HO- KNCSS product. 

These results indicate that adjustments are necessary in 
the way values (nr the model parameters are derived. For 
instance, currently the values for the parameters are aver- 
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aged from a heterogeneous population of previous software 
projects. Fixing this problem means that if we want to 
estimate the QA duration for project X I hen we must base 
the model parameters on projects similar to project X. Proj- 
ect characteristics such as complexitv', team experience* 
and the development language must be formally factored 
into any early estimates of QA duration. 

Field Failure Results 

Although the modeling process is helping to monitor 
progress through system testing and is aiding in the release 
decision, based upon limited defect data from our field 
defect tracking system it appears that the cun^es may be 
overestimating the number of defects customers will dis- 
cover by as much as iorty times. ^ However, it is likely that 
only a portion of the actual field failures find their way 
into the tracking system. 

Our expedence tesling one firmware project that was an 
enhanced version of an old instrument puts an interesting 
perspective on estimated residual defect densities. This 
particular product had been shipped for ten years at an 
average volume of over 200 units per month. Since a market 
apportunity existed for an updated version of thai product, 
both hardware and firmware enhancements were incorpo- 
rated into a new version. During system test» an obscure 
defect in a math routine was discovered that had nut only 
existed in the original product since introduction but in 
several other products shipped over the last ten years. To 
the best of our knowledge . no customer or HP personnel 
had previously found that failure, Its existence was furl her 
argument that the information coming liack from the field 
is not giving a true perception of residual defect densities. 



Nat only do customer obsen'ed failures go unreported, but 
it is highly likely that some failures will never be encoun- 
tered during operation. It was reassuring to note that LSlD*s 
current testing process was uncovering defects so obscure 
as to be unobsen'able in ten years of field use* 

Conclusion 

With data collected during system integration testing, 
we have been able to use a software reliability model to 
estimate total testing effort and aid in assessing a prore<:t's 
readiness for release to customer shipments. Although the 
model appears to be somewhat robust to its underlying 
assumptions, future success will depend upon the integra- 
tion of customer representative testing techniques into our 
existing testing process. In addition, there remains the chal- 
lenge of using the model t(J estimate test duration before 
system integration begins. This will require a ihurough 
analysis of data on past projects and key information on 
the current project to derive better early estimate.s of the 
moders parameters. Our ultimate objective remains to 
achiev^e validation of the modeling process through accu- 
rate field failure data. All of these areas will continue to 
be investigated because they are important in determining 
project schedules and estimating product quality. 
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Comparing Structured and Unstructured 
Methodologies in Firmware Development 

Structured methodologies have been promoted as a 
sotution to software productivity and quality problems. At 
HP's Logic Systems Division one project used both 
structured and unstructured techniques, and collected 
metrics and documented oberservations for comparing the 
two methodologies. 

by Wilfiam A. Fischer Jr. and James W. Jost 



STRUCTURED METHODOLOGY m softwaa^e de- 
velopment tends to be very much a religious issue 
with many practitioners. They are either strongly 
for or against it, but they can point to very little data that 
supports their point of view. The purpose of this paper is 
to present some objective and subjective data on the relative 
merits of structured and unstructured (traditional) method- 
oiogies. 
The data tor this comparison came from the development 



of a medium-to-large-scale firmware project at HP*s Logic 
Systems Division. It was the first project at thi.-:. divisiun 
to use structured methods. The project consisted of an em- 
bedded 68000 microprocessor design^ coded in C and using 
a C compiler running on the HP-UX operating system. The 
firmware consisted of about 47 KNCSS (thousand lines of 
noncomment source statements] of new code and about 12 
KNCSS of reused code. 

At the start of the project, a goal was established to use 



m 



MODULE MAIN PROGRAM,B.C,D,E,F,H; 
SYSTEM G; 
EXTERNAL I; 
HARDWARE J; 
DATA K: 
RECURSIVE L; 
MAIN PROGRAM 
BEGIN 

B(TO,PARM); 
C 
BEGIN 
D 

BEGIN 
I; 
J: 

END; 
E: 

END; 
F( FROM.PARM); 
*L0OP 
BEGIN 
G; 
K; 

END; 
*COND L(T0_PARM/FR0M_PARMJ^ 
END; 
(b) 

Fig. t. (a) An example of a simple hierarchy chart showmg the riifferent types of modules. 
MAfN PROGRAM, B.C^D^Ef, and H (r^ot called) are modules. G /s a system module. I Is an 
external module. J is a hardware module. K ts a data module. L /s a recursive rnodule. Module 
names can be up to 32 characters long. HCL draws each module name on up to three lines 
withm a symbol, (bj The commands used to create t^e hierarchy chart. 
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stnictured metiiodologies^'^ to improve the software de- 
velopment process and increase product quaJit\\ but the 
decision of whether to use structured methods on each 
subproject was left up to the individnal engineers. As a 
consequence, three of the suhpro|ects were developed 
using structured techniques and the other six used Iradi- 
tionaJ methods. Software designers using structured lech- 
niques did their analysis using data flow diagrams (DFDs) 
and structure charts for their design. They also did inspec- 
tions on most of the analysis and design documents. HP 
Teamwork/SAj a graphical tool designed for structured 
analysis that also performs consistency checking, was used 
for structured analysis. HCL [Hierarchy Chart Language)^ 
was used for structured design. HCL is an internal HP tool 
that plots a program structure from Pascal-Uke statements 
(see Fig. 1). 

For engineers who used the traditional methods, the 
analysis phase typically consisted of creating a written 
specification. Informal design methods were used and cod- 
ing was started earlier in the product development cycle. 
Inspections were generally not part of this process. 

This was not a scientifically designed experiment to de- 
termine the better of the two methods. Rather, we simply 
collected data on the two groups of engineers as the project 
developed. As such* our data suffers from many of the 
common problems that beset unplanned experiments. 
However, since the data was collected from a single work 
group, many of the variables that are factors in most com- 
parisons of project experiences have been eliminated. For 
example, lines of code and time expended were measured 
and reported in the same w^ay. Intangibles, such as work 
environments computer resources* complexity of task, and 
management attitudes are also identical. 

Many experts say the most important variable influenc- 
ing progriimmer quality and productivity is individual 
skill. The difference in the experience level belween our 
two groups was not su bstan I iaL However, the tmstructured 
group was more highly regarded by management than the 
structured group, it is possible that those in the unslruc- 
tured group had already demonstrated winning techniques 
for which they had been rewarded, and so they were reluc- 
tant to try newer methods, while the structured group was 
more willing to try new methods to improve their overall 
skill level. 

Data Collection 

The data in this report was collected from the following 
sources: 

■ Engineering Time. The time spent by each engineer was 
reported to a central data base on a weekly basis. The 
time the engineer spent doing such things as system 
administration^ meetings, and classes was not included 
in the reported time. Only time spent in aiialysiSt design* 
test, or coding on the engineer's primary software project 
was included. Time data was reported by the Individual 
engineers. 

■ Defect Data. The defect data was collected by DTS (defect 
tracking system)."* an internal defect tracking tool. De- 
fects were reported from the beginning of system inte- 
gration testing. Tlie defects discussed in this paper are 
only unique, reproducible defects. Duplicate, nonre- 
pniducible defects, operator errors, and enhancements 



were not included in the defect count. Defect data was 
reported by the indi\idual development engineers and 
the formal and informal testers. 

m KNCSS and Complexity. All the KNCSS counts and 
McCabe's Cyclomatic Complexity metrics were com- 
puted by an internal tool called Ccount. 

M IDestgii Weight. Design weight,^ a measure of the effort 
of coding and testing, was calculated from the C code 
by an internal tool. This tool counts all the decisions 
and the unique data tokens that are passed into and out 
of a function. Any system calls {e,g.. printf) are not in- 
cluded in the token count. 

Comparison Results 

To facilitate the comparisons, the project was broken 
down into subprojects that closely corresponded to the 
efforts of individual software designers. Each subproject 
was then categorized as being representative of either the 
structured or the traditional methodology. The results of 
the two methodologies were compared on the basis of pro- 
duction ty and quality. Development effort, manageability, 
communication, and reusability were the criteria used for 
productivity measurement. TheFURPS [an acronym stand- 
ing for functionality, usability, reliability, performance, 
and supportability) model was used as the basis for compar- 
ing quality. 

We have used metrics to evaluate these factors wherever 
possible. Where metrics did not exist we have presented 
the subjective views of the authors who observed the proj- 
ect team throughout the development of the product. 

All statistical tests referenced in this paper were made 
at the 95% confidence level. Since the sample size used 
for the comparisons between the structured and traditional 
methods is small, all conclusions are very tentative. 

Productivity 

Achieving productivity in software development re- 
quires using tools and techniques that yield optimal return 
on development money. 
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Fig, 2, Engineering hours versus des^gr} wetght. Design 
weight is a measure of the effort of coding and testing. It is 
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Developinant Effort 

Pro) eel managers tend to be most concerned with the 
compleHon of a project on schedule. Consequenlly, one of 
the first questions asked by managers considering the use 
of structured methods is whether it will shorten the de- 
velopment cycle. 

Since b learning curve was involved with the structured 
method s» we expected that since they were being used for 
the first time, the strur:tured team would be less productive 
than the unstructured team- To verify this assumption, two 
measures of productivity were used, design weight per en- 
gineering hour W'Orkcd and NCSS per enghu^erinj^ hour. 
These graphs are shown for the various subprojecls in Figs. 
2 and 3. It can be seen that the three structured subprojects 
have lower productivity than the unstructured subprojects- 
A statistical test using genera 1 linear hypothesis tech- 
niques*' showed that indeed a statistical difference did exist 
in the productivity rates between the two methodologies. 
Using the structured methods, the average productivity rate 
was 2. 40 lines/engineering-hour, while using traditional 
methods resulted in an average productivity rate of 6,87 
lines/engineering-hour, 

A central problem with analysis performed with DFDs 
is that it is an iterative process- The DFDs were leveled 
until the iowest-leveJ bubbles could be completely de- 
scribed in a minispecification of about one page as rec- 
ommended by the methodology. This is a rather subjective 
requirement and we discovered that every designer using 
DPDs for the first time leveled them more deeply than 
required. A project review also confirmed that too many 
intermediate layers were created to keep the complexity 
per page of DFDs to a minimum. At the project postmortem , 
it was discussed that a major contributing factor to lower 
productivity with the structured methods was the lack of 
an on-site expert. Consultations were needed at various 
points in the analysis and design phases of the project to 
verily the proper application of the techniques, 

ManageabMity 

The structured work habits seemed to help the project 
manager and engineers understand the software develop- 
ment Ufe cycle better. Designers had a l^etter idea w^hen to 



end one phase of the project and begin another, helping to 
make their own process better understood and easier to 
manage. Figs, 4 and 5 show the times spent in various 
project life cycle phases for structured and traditional 
methodologies ^ respectively. The structured methods 
graph shows cleaner, better-defined phase changes. These 
clear phase changes aid the management planning process 
by creating a better understand jjig of the state of a project. 
Plots showing the same data as Figs, 4 and 5 were done 
for each engineer, and these individual plots showed the 
same characteristics. 

The regularity of the structured life cycle can be used to 
nn prove st:tiedule estimates. Once the percentages of time 
spent in the phases are historically established, the time 
taken to reach the end of a phase can be used to project 
the finish date. For example^ if it takes four months to 
complete the analysis phase and the liistorical figures indi- 
cate that 33 percent of the time is spent in analysis, then 
it could be estimated that the project would be completed 
in eight more months. 

It is important to measure the progress of projects against 
the established schedule/ Keeping track of the actual com- 
p! el ion time of each phase can provide an independent 
verification of established scheduling methods. If problems 
in meeting schedule commitments are uncovered, correc- 
tive action, such as adding additional resources, can be 
applied to the project. 

Taking a project through the system test phase is unpre- 
dictable. The time it takes to complete the formal abuse 
testing is dependent on the quality built into the product, 
If the product is well-designed and coded, less time is spent 
repairing defects found during abuse testing and during 
the completion of formal regression tests. A reduced testing 
phase can shorten the overall project development time. 
More important, if fewer defects are embedded in the prod- 
uct, less time will be spent in the maintenance phase, w^hich 
can consist of 50% of the project's overall cost, Fig. B shows 
a comparison of the limes spent in the various development 
phases for the project. The graph indicates that a greater 
percentage of time was spent in the analysis and design 
phases with the structured methods. However, a very small 
percentage of time in comparison to the traditional methods 
was spent in testing. 
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Reusability 

Reusability of code is a major productivity issue at HP 
in general. Several of the products at our division have 
ah-eady reached reuse figures of 50% and we are working 
to increase that figure in the future. Experts suggest that. 
Oil average* reusing a piece of software requires only 20% 
of the effort of writing it from scratch. Thus, any activity 
that leads to greater reuse of code will have major benefits 
for productivity'. Code developed using structured tech- 
niques encourages more reuse in the future because it has 
better documentation on each function, hi addition, the 
interface between that function and the outside world is 
more clearly stated by the structured design documenta- 
tion. 

A major concern is the maintenance of the software 
documentation. Documentation that is not kept up to date 
is of less value than no documentation, since it can be 
misleading. There is a very w^eak connection bet w^een struc- 
tured analysis and structured design. This makes it difficult 
and often impractical to keep the structured analysis up 
to date because of changes in the design and coding. The 
structured team chose not to keep the structured analysis 
documentation up to date w^hen design and coding changes 
were made later in the projectn This takes away some of 
the documentation benefits provided by the structured 
methods. As back annotation methods are incorporated 
into the tools, this deficiency will be corrected. 

Communication 

One of the positive attributes cited in the literature about 
the structured methods is that they serve as a good com- 
munication tool between team members. The project saw 
some of this benefit, but not as much as was originally 
expected. Structured methods are not a team communica- 
tions panacea. 

Each software designer on this project was assigned a 
nearly autonomou.s sub project. Although there were in- 
teractions between the sub projects, the subprojects were 
defined to minimize these interactions. The structured 
analysis documentation for each subproject was large 
enough to cause difficulty in obtaining the number of de- 
sign reviews that were necessar>'. For another team member 
to understand the analysis, much additional verbal expla- 



nation was required. Howevefp the structured analysis was 
vers^ useful to the developers of that analysis in fully under- 
standing their own subprojects. 

Fig. 7 shouts the staffing profile of hardware and softw^are 
engineers during product development. The staffing of soft- 
w^are engineers lagged behind hardware engineers because 
of an initial underestimation of the software task and the 
shortage of softw^are engineers. As a result, there were some 
problems discovered during the hardw^are/software integra- 
tion that cost valuable schedule time. Since w*e were de- 
^"^e loping a system that consisted of both hardware and 
software, it would have been beneficial to have included 
the hardware interfaces into the structured analysis. This 
capability would have aided the hardware/software inte- 
gration by providing additional communication links be- 
tween the hardware and software groups. 

There is probably a strong benefit in communication with 
structured analysis if the whole project team uses the 
methodology to develop the same system model. This en- 
ables each team member to have a better understanding of 
the product^s requirements, and helps designers under- 
stand the task in the same way* 

Quality 

High product quality improves customer satisfaction, 
decreases maintenance costs, and improves the overall pro- 
ductivity of the development team. The FURPS model was 
used as the basis of comparison between the two method- 
ologies. 

Functionaltty 

Functionality can best be measured by customer accep- 
tance of the product The product resulting from this project 
is still in the early stages of its product life and customer 
acceptance of its functionality is yet to be determined, ft 
w^ould also be difficult to separate the functianaiity of the 
code generated by the two methods. 

However, we believe that the rigor required by the struc- 
tured methods is an important contribution to the develop- 
ment life cycle. Structured analysis was a valuable exercise 
for the software designers in obtaining an understanding 
of the customer requirements. Although the structured 
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analysis was not used directly as a communication tool, it 
helped the software designers identify the correct questions 
to ask when issues related to functionality were addressed* 
These benefits assist the product definitioOp and enhance 
the chances that the product will meet all user expectations. 
We also believe that the entire team benefited from the 
efforts of the structured group. Since much of the initial 
product definition was performed using structured 
methods, software designers joining the project later bene- 
fited greatly from the initial structured w^ork. 

Usability 

The look and feel of the user interface was of prime 
importance to the project. Structured analysis was useful 
in breaking up the interface functionality into its elemental 
commands. How^ever, manual pages combined with verbal 
discussion proved to be more effective in defining the de- 
tails of the interface. 

Neither methodology appeared to enhance usability 
more than the other. However, the structured methods can 
help the software designer define user interface functional- 
ity. Another method that appears to be a better method for 
communicating and testing of user interface Ideas is rapid 
prototyping. This method was not used on the user inter- 
face. 

Reliability 

Reliability is measured by the numbern types, and fre- 
quency of defects found in the software. The defects found 
in code created using the structured methods were com- 
pared with those using the traditional methods. Table I 
outlines the defect rate normalized to defects per NCSS for 
both prerelease and postrelease defects. Prerelease defects 
were found during formal abuse testing, casual use by other 
individuals, and the code's designer. Postrelease defects 
include prerelease defects and the defects found in the first 
four months after product release. All the postrelease de- 
fects were found internaily either by abuse testing or by 
casual use. No customer defects had been reported at the 
time of this study, 

Table t 
All Defects 



Unstructured 
Structured 



Prerelease 
Defects/NCSS 

0.0041 
0.0036 



Postrelease 
Defects/NCSS 

0.0052 
0.0050 



Although the structured code shows a slightly lower de- 
fect density than the unstructured code, the differences are 
not statistically significant [using a statistical test that com- 
pares two Poisson failure rates^]. 

Low-severity defects are considered to be caused by typ- 
ical coding errors that occur at the same frequency, inde- 
pendent of the analysis and design methods used. Thus, 
using ail the DTS defects is not truly indicative of the 
underlying defect density. Another w^ay of characterizing 
the defect density is to look only at severe defects, those 
classified as serious or criticai. Table II examines these 
defects. 



Table 11 




Serious and Critical Defects 


Prerelease 


Postrelease 


Defects/NCSS 


Defects/NCSS 


Unstructured 0.0009 


0-0010 


Structured 0.0007 


0.0008 



Again, the structured code show^s a slightly lower density 
but the difference is not significant* We knew that the code 
designers' rigor in logging defects that they found them- 
selves varied a great deah Since this might affect the quality 
results, we examined only the defects logged during formal 
abuse testing. It was again found that there was no statistical 
difference between the methodologies. 

Our theory, developed from these results, is that the final 
reliability of a product is mainly a function of environmen- 
tal factors, including management expectations and peer 
pressures. For example, reliability can either be designed 
in at the beginning using structured methodologies or 
tested in at the end of the project with thorough regression 
tests. 

Performance 

In the design of structured modules, one Is encouraged 
to reduce complexity of the modules by breaking up func- 
tionality. This results in more function calls, and increases 
the processing time of critical functions. 

The software for this project had critical performance 
requirements for communication rales. The processing of 
the bytes of data by the firmware was the critical path. The 
critical functions had to be receded using in-line assembly 
code. Altbougb structured methods were not used on this 
communication firmware, it is our opinion that the struc- 
tured methods as used on this project w^ould not have 
helped to identify performance-critical functions or to im- 
prove the performance of these functions. 

A newer form of structured analysis^ has been developed 
and is based on modeling of the problem data as the first 
step. The data is represented in an information model with 
the state control diagrams showing the data control. This 
may in fact prove to be a better model for real-time appli- 
cations that have critical performance requirements. 
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Attribute 


Structured 


Unstructured 


Productfvfty I 


Development Time 
ManageabilitY 
Reusatiility' 
Communi cation* 


longer 
better 

easier to reuse 
slightly better 


shorten 
worse 

harder to reuse 
slightly worse 


Quality 


Functionality" 

Usability' 

Reliability 

Performance' 

Supportability 


better 

no difference 

no difference 

worse 

better 


wprse 1 
f^o difference 
no difference 
better 
worse . 



*These items had no ol^jectEve metrics kept on ttiem. 

Fig. 8. Summary of comparison between structured and un- 
structured methodologies 

Supportabillty 

A large part of the costs of software development and 
maintenance can be attributed to maintenance activities. 
Software that is supportable will have lower maintenance 
costs. Maintenance not only refers to the repair of defects, 
but also to software enhancement and modification. 

One factor that relates strongly to software supportability 
is the complexity level of each function. Functions with 
lower complexity tend to make it easier to modifj^ and 
repair defects. For this project, the average complexity of 
each module of the structured code was less than the aver- 
age for the unstructured code. Table HI shows a summar^^ 
of the complexity statistics for the two code types. 

Table HI 
McCabe'5 Cyclomatic Compiexlty 



Mean 

Number of Functions 


Structured 

5.7 
268 


Unstructured 

7.2 
656 



Modules with a cyclomatic complexity greater than 10 
may be too complex and should be reviewed for possible 
restructuring/' Only 13% of the structured code had a com- 
plexity greater than ten, while the unstructured code had 
20%. A statistical test (comparison of two binomial frac- 
tions^) showed this to be a significant difference. 

The discipline of the structured methods helps the de- 
signer to write consistent documentation that can be used 
by those who must support the product. 



Conclusions 

Our analysis of structured and unstructured techniques 
produced mixed results. It was apparent that the structured 
methodologies on this project did not provide improve- 
ment in the project development time. In fact, a longer 
development time was measured- This w^as partially be- 
cause of the lime required for learning the structured 
methodologies. Howeven the manageability of designers 
using structured techniques is higher because of weil-de- 
fined development cycles. Also, the structured code ap- 
pears to be more reusable, so it should improve the produc* 
tivity of future projects reusing this code. 

In this project, structured methods didn't appear to im- 
prove the reliabilit\'^ of the final code significantly. It is our 
opinion that reliability is more a function of the team's 
environmental factors. The structured code appears to be 
more supportable since module complexity was lower. 
Structured methods do not appear to be a major benefit Ln 
developing code where performance factors are a main re- 
quirement. No significant benefit was seen for the areas of 
functionality and usability except in those projects where 
the techniques were used to enhance communication of 
the product design and specification. 

Some aspects of the structured method olog}' w^ere disap- 
pointing, How^ever, w^hat was most important for our de- 
velopment team was the discipline that the structured 
methods added to the early phases of the product defini- 
tion. We feel the results of the structured methods are posi- 
tive enough to continue using these methods on future 
projects. Fig. 8 summarizes the results of the comparison 
of the tw^o methodologies. 
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An Object-Oriented Methodology for 
Systems Analysis and Specification 

A methodology is proposed that enables analysts to model 
and specify a system's data, interactions, processing, and 
external behavior before design. 

by Barry D. Kurtz, Donna Ho, and Teresa A. Wall 



AS SOFTWARE SYSTEMS BECOME LARGER and 
more complex, the task of systems analysis con- 
tinues to increase in effort and difficulty. Tradi- 
tional methodologies for systems analysis sometimes fail 
to meet the analyst's expectations because of their limita- 
tions in properly capturing and organii^ing all of the infor- 
mation that must be considered. Object-oriented systems 
analysis (OSA) Is an approach to systems analysis and 
specification that builds upon the strengths of existing 
methodologies and , at the same time, addresses their weak- 
nesses. This paper describes the basic concepts of the OSA 
methodology. 

Systems Analysis 

A system is an organized collection of people, machines, 
procedures, documents, data, or any other entities interact- 
ing with each otiier to reach a predefined goal.' Analysis 
is the study of a problem, prior to taking some action/ 
Systems analysis is the study of a collection of related and 
interacting objects (existing or proposed] for the purpose 
of understanding and specifying its data, processing, and 
external behavior. 

Object-oriented systems analysis (OSA) is a systematic 
approach to the study of a system problem. The foundation 
of OSA's conceptual model is the set of components or 



objects in the system. The study of these objects is or- 
ganized and conducted in a manner that does not un- 
necessarily constrain implementation. In fact, we believe 
that designs based on OSA specifications can be procedure- 
oriented or object-oriented with equal success. 

Objects in the OSA Methodology 

An object is an abslracliun of entities or concepts that 
exist in or are proposed for a system. For example, the 
rectangle in Fig, 1 depicts an OSA object called Jet Pfane. 
This object represents a class of things where each member 
has the attributes and behavior of a jet plane- Two possible 
members of this class are a jet fighter and a jet passenger 
plane. In object-oriented terms, each member is called an 
instance of the object class. Fig, 2 further demonstrates the 
community of attributes and behavior required of each 
member of the object class. The circle on the top in the 
figure represents all of the attributes and behavior of a jet 
fighter. The circle on the bottom in the figure represents 
all of the attributes and behavior of a jet passenger plane. 
The shaded area represents the common attributes and be- 
havior that allow each plane to be a member of the jet plane 
object class. 

The key elements of a system are the objects that exist 
in the system, the data that must be processed, and the 
desired behavior of the system, Traditinnal methodologies 
offer two basic approaches to modularize these elements; 
a function- oriented approach w^hich organizes the analysis 



T 




Fjg. 1, An OSA object. Each member of this object class has 
the behavior and attributes of a jet plane. 



Set of Common Aflributes 
and Common Behavior 




Fig. 2. Commonatlty of attnbutes and behavior of objects of 

class Jet Plane* 
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model through a hierarchy of functions, and a data-oriented 
approach which emphasizes a hierarchy of data structures. 
OS A Imposes a natural modularization on ttie system 
model through the emphasis on objects (see Fig, 3), Each 
object has associated attributes (data) and behavior (legal 
states and functions). An object in the OS A system model 
has a one-to-one correspondence with an actual object in 
the system. This provides an easy mapping between the 
analysis model and the components of the system under 
study. 

Collecting Preliminary Specifications 

Before attempting to construct an OS A system model, it 
is important to coiJect or generate a preliminary list of 
specifications. These specifications are gathered from dis- 
cussions and intennews with users and managers of the 
system under study. The preliminary specifications are 
rather informal and are usually documented with natural 
language text and hand-drawn graphics. These specifica- 
tions should answer most of the following questions that 
may be posed by the analysis team: 

B What is the subject of the analysis? Is it an existing sys- 
tem, a proposed system, or a combination of both? 

■ What are the specific problems that need lo be solved 
and what are the objectives for the solution? 

n What are the logical boundaries of the system study? 
Does the entire system need to be studied in detail or 
can a smaller subset be studied? 

■ What are the known constraints of the system? Are there 
special performance constraints, interface constraints, 
hardware constraints, and so on? 

■ What are the major components of the system? What 
needs to be done with each component? What should 
each component do in the system? 

The specific format of the preliminary specif ications may 
vary from project to project. The important factor here is 
that the preliminary specifications are collected before con- 
struction of the formal OSA specifications. The following 
is a small example of a prelim inaiy specification that an 
analyst might use to gain a basic tinderstanding of a system 
problem. 




Is Held 
By 



O 



Holds 



W 




A uending machine must be produced foi dis- 
tributing products to customers. The customer 
begins the vending process hy depositing a token 
in the machine's com receptacle ^ The token must 
be checked for size, iveight, thickness^ and type of 
edge. VoUd tokens (coins) are quarters* dimes, or 
nickels, and an>lhjng else is considered a slug. 
Slugs are rejected and sent to the coin return slot. 
When Q valid coin is accepted by the machine and 
sent to the coin box. the amount of current customer 
payment is incremented by the value of the coin. 

Each product dispenser contains zero or more 
products. AU of the products m one dispenser have 
the same price. The customer's product select ion is 
made by identifymg the dispenser to be activated. If 
the dispenser is not empty and the current customer 
payment equals or exceeds the cost of the product, 
the product shoukl be dispensed fo the product deliv- 
ery slot and any change due returned to the coin 
return slot. If the dispenser is empty, coins equaling 
the current customer payment should be dispensed 
to the customer in the coin return slot. 

if the customer payment is less than the price of 
the products in the selected dispenser, the machine 
should wait for the customer to deposit the proper 
payment. If the customer decides not to make a 
selection the current amount deposited should he 
returned. 

Building Object-Relationship Diagrams 

After preliminary specifications are gathered for the 
major components of the system, the analyst can begin 
building object-relationship diagrams [ORDs). The ORD 
portion of the analysis provides a formal repository for 




O 




Obieoil Is related by ™u to Obifrei2 . 
O&ietia is related by 'eS2 to Ob|ecii - 
M2 designates how many instances of Otij&ct2 are 



associated with each instance of Obrecil 
Ht designates how many instances of 



Otojecll are 



Fig. 4. An exampfe of an obiBCt-relatianship diagram The 
coin box holds many (M) coins. 



assoctat&d with each instance of Obje<:t2 
Fig. 5. Formal definitions in object-reiationsiiip diagrams. 
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answers to the following questions: 

■ What are the components of the system (both abstract 
and concrete)? 

■ What are the important formal relationships between 
the system components? 

The first step for producing ORDs is to make a list of the 
nouns contained in the preliminary specification. These 
nouns are classified by the analyst as objects (system com- 
ponents], attributes of objects, or synonyms for other ob- 
jects or attributes. Synonyms m:ust be eliminated and attri- 
butes must be associated with existing objects. From our 
vending machine example, some of the nouns that repre- 
sent objects are token, coin receptacle ^ slug^ coin, coin box, 
nickel, dime, and quarter. Also from the same example, 
the nouns that would be considered attributes include 
weight, thickness, and price. 

Objects are depicted in object- relationship diagrams by 
the abject name enclosed in a rectangle. An ORD showing 
a relationship between coins and a com box is shown in 
Fig. 4* The ORD in the figure states that a coin box holds 
one or more coins. The capital M in the figure is shorthand 
for one or more. A relationship line without an M signifies 
that the relationship is one-to-one. ORDs are similar in use 
and meaning to entity-relationship diagrams.^ The formal 
meaning of ORD relationships is shown in Fig. 5, and Fig. 
6 shows a partial ORD for the vending machine example. 

Object-relationship diagrams have a subcomponent called 
concept diagrams. Concept diagrams provide a quick method 
lor capturing the information relationships and interac- 
tions between objects. For example, Fig. 7 shows a concept 
diagram stating that a coin receptacle needs to know the 
thickness of a token. A dotted arrow depicts an informa- 
tional relationship between objects. The information may 
be an attribute of an object or an event that another object 



Coin 




SQurce 



Oesti nation 



Fig. 7. An example of a concept diagram. The coin recepta- 
cle needs to know thickness of the can. The concept diagram 
shows the informatfona! refationshfp and fnteractions between 
objects. The dashed tine indicates information flow. 



Fig. 6. Partial object-refattonship 

diagram for the vending machine 
example, it shows that a token en- 
ters the vending machine through 
the coin receptacte. and that a 
token may be a can or a slug. The 
only objects considered to be a 
iegitimate coin in the system are 
mckeis. dimes, or quarters. 



needs to know about. 

Fig. 8 shows a concept diagram that states that a customer 
deposits a token into the vending machine, A solid arrow 
is used to show an action that an object performs upon 
another object. Since informational relationships and oh- 
ject interactions are relatively easy to extract from the pre- 
liminary specifications, concept diagrams may be used fre- 
quently at the beginning of the analysis. 

Natural Language Object Descriptions 

The object-relationship diagrams identify classes of com- 
ponents in the system and formal relationships between 
the components. The next step is to provide more informa- 
tion about each object and Its attributes. The object descrip- 
tion is documented with natural language text and contains 
the following information: 

■ Name. The name of the class of system components rep- 
resented by the object. 

■ Description. A brief description of the general charac- 
teristics for instances of the object. 

• Assertions, Things that must always be true regarding 
attributes or behavior of instances of this class. For exam- 
ple, an object may need to be kept at a certain temperature 
to ensure the desired behavior, 

■ Attributes. Required attributes for each instance (such 
as identifiers, status variables, etc.). The domain or legal 
range of values for each attribute must be specified here. 
The specific format of the object description may vary 

from project to project but it should include at least the 
above information. Fig. 9 shows a natural language descrip- 
tion for the vending machine example. 

Building Behavior Specifications 

Using the foundation of object-relationship diagrams 
and individual object descriptions, the analyst builds a 
behavior specification for each object. Since internal be- 
havior is usually a function of implementation, the analyst 
concentrates on the external behavior of each object, Exter- 



Customer 



Deposits 




Fig. B. A concept diagram that shows an action of one object 
on another The soiid fine indicates action. 
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Object Name: Tokwi 

Description: 

A token is s^methtng iNe custornet deposits into a vending mactiin 
to purchase a protiucL 



The diameter an<j thJcKJiess are sm^l enough to allow it to fit in the 
com stol 

Attributes :. 

Diameter Domain: diameter height of covn slot 
Thickness Domam; thickness wtdlh of coin slot 
Weight Domain: not yet detined 
Edge Type Domain: {smooth, serrated) 



Fig, 9* A naturai language description for a token object in 
the vending machine example 

naj behavior has three components: 

■ The externally perceived states or conditions of each 
object 

■ The actions that jiiust be performed on each object 

■ The actions that each object must perform. 

Each of these components may be specified by natural 
language text. However, experience has shown that such 
an approach leads to ambiguity and coEfusion. To help 
ensure consistent interpretation of behavior specificaEionSt 
a more formal method is required. 

There are several existing formal methods for describing 
external system behavior. These methods include finite 
state machines, decision tables, Petri nets* R-nets, precon- 
ditlon/action/postcondition tuples and tMhers.*'^ The fol- 
lowing criteria were used to choose a behavior specification 
method for OSA: 

■ The method should encourage the analyst to Identify 
and classify system uomponents before specifying be- 
havior. 



Object 
Name 




Transitton — * -- 



Token _Depc5ited 




Required- 
Action 




^ Mo«/& Token Tq 
Coin Box 



flfl|scted 



Riove Token Ta 
Rfliurn Slot 



m 



Ftg. 10. State net showing the behavior of a token object 
When a token is deposited, it transfers from the customer's 
possession to the coin receptacle if the token is a valid coin, 
it is tfansferred to the com box as a coin. Otherwise, a rejected 
token is returned to the return slot as a slug. 

■ The method should allow the analyst to specify an object 
that exhibits multiple action conditions (concurrent ac- 
tivities). 

■ The method should provide for high traceability between 
the actual system components, object-relationship dia- 
grams, and behavior specifications. 

The behavior specification technique developed for OSA 
is called state nets. State nets are based on a restricted form 



Build Object-Relationship 

Diagrams for System 

(Include Concept Diagrams) 



Complete a 

Natural Language 

Specification for 

Each Object 



\ 



Collect Natural Language 

Specifications Obtained from 

User Manager Interviews 



;^ 




Simulate Syste^n Behavior and 
Correct Specification Errors 




Build Behavior Specifications 

for the System Starting at the 

Object LeveJ 




Fig. 11, Process fiow for the OSA 
methodology and the interactions 

between tiie object and method- 
ology steps. 
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of Petri nets. There are several possible configurations of 
state nets that use the full power of Petri nets for expressing 
concurrent activities. For this overview, hov^^ever, we will 
limit the discussion to a model similar to a finite state 
machine. 

A sample state net representing some of the states and 
actions required for a Soken is shown in Fig* 10. The states 
or conditions of the token object are represented by ovals 
in the figure. The states are typed with the name of the 
object being specified (token in this example). Labeling ttie 
state with the object name means that instances of that 
object may exhibit the specified state. The transitions be- 
tween states are represented by short horizontal bars. The 
labels next to the transitions specify the eveots that cause 
instances of the object to transition from one state to 
another. In Fig. 10, the Token„E)eposited command causes the 
machine to transition from the In Customer Possession state 
to the In Coin Receptade state. The actions required to estab- 
lish a new state are shown as labels on the input arrow 
connected to the new state. For example, the state net di- 
agram states that the action Move Token To Return Siot must 
be performed before the mat:hine is in the In Return Slot state. 

System Behavior Simulation 

Once the statt^ riets for the system have been completed, 
the analysis team is prepared to simulate executions of the 
system. The simulation is performed using state nets to 
follow conditions and states exhibited by the system. The 
object-oriented nature of state nets aids in hiding complex- 
ity during simulations. This allows the analysis team to 
achieve good behavior coverage even when automated 
simulation tools are not available. 

During system simulation, several classes of errors may 
occur that require modifications to the current OSA model 
or specifications. Simulation will aid the analyst in discov- 
ering the following problems: 

■ Missing or ambiguous information in the preliminary 
specification 

■ Missing or incorrect object-relationship diagrams 

■ Missing objects or incorrect object descriptions 

■ Missing or incorrect behavior specifications. 



Any of these errors will require another pass through 
one of the analysis phases previously discussed. The object- 
oriented nature of the OSA model enhances traceabillty to 
the components of the system related to a particular error. 
Fig. 11 shows the process flow for the OSA methodology 
and shows possible interactions between the various 
methodology steps. 

Conclusion 

The contributions of the object-oriented analysis 
methodology include: 

■ It provides tools for capturing the key elements of a 
system — components^ data, and behavior. 

■ It emphasizes analysis before design. 

■ It supports a natural modularization scheme by organiz- 
ing the entire analysis around objects, 

■ Its system model provides high traceability between the 
model components and the actual components of the 
system under study. 

The methodology is currently under research and verifi- 
cation. It has been applied to several problems involving 
hardware and software components with encouraging re- 
sults. A prototype tool is being developed to aid analysts 
in processing textual specifications and capturing the 
meaning of systems problems using the OSA methodology. 
The goals now are to finish the tool, develop training ma- 
terials, and continue to verify applicability of the methodol- 
ogy to real -world projects. 
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VXIbus: A New Interconnection Standard 
for Modular Instruments 

This standard will allow users to mix modules from differerjt 
manufacturers in a system contained in a single mainframe. 

by Kenneth Jessen 



THE GOAL OF THE VXIBUS Is to provide a techoi- 
cally sound standard for modular instruments that 
is based on the VMEbus [defined later) and is open 
to all manufacturers. It is a specification for interconnecting 
and operating various modules from a variety of manufac- 
turers v^'ithin a single mainframe to satisfy the need for 
high-performance, high-density instrumentation. 

Users are able to select from four module sizes and are 
free to choose modules and a mainframe from different 
suppliers based on price and performance. The VXIbus 
standard ensures compatibility of all the elements within 
a VXIbus system. For example, a user may find that a par- 
ticular digital multimeter module offers the best combina- 
tion of price and measurement capability for a particular 
job, but that the best function generator for the application 
comes from another manufacturer. The user may then select 
a third manufacturer for the mainframe. This amounts to 
unprecedented flexibility in the creation of an instrumen- 
tation system, 

VXIbus Evolution 

Many years ago there were only bench instruments. They 
had front panels for control and displays for data output. 
As computers entered the picture, digital interfaces (such 
as binary -coded decimal) came into use. These were typi- 
cally offered as options to the basit. bench instrument and 
varied from instrument to instrument and from manufac- 
turer to manufacturer. In the early 197f}s. a better interface 
standard ^ the HP Interface Bus. or HP-IB, was developed 
by Hewlett-Packard. This became an industry standard 
(IEEE 488, lEC 625) and is currently used by a wide range 
of manufacturers. 

Instruments continued to retain their front-panel con- 
trols, but as applications moved toward automatic test and 
measurement systems, the need for front panels di- 
minished. Instruments began to appear on the market either 
^vithout front panels or with detachable front panels used 
only during initial setup. 

To reduce the si^;e of a test system, modular instruments 
have grown in popularity, but until recently there were no 
industry standards for such systems, so there was no pos- 
sibility of mixing modules from different manufacturers 
within a mainframe. Often modules from a manufacturer 
were not compatible with different mainframes from that 
same manufacturer. 

In 1979, Motorola Semiconductor Products Corporation 
published a description of what it called the VERSAbus. 
il defined a computer-typf^ backplane and cards. Eurocard 



board sizes were proposed and the VTRSAbus-E version 
was renamed the VMEbus. In 1982 the iEC (International 
Electro technical Commission) proposed thai the VMEbus 
be accepted as an international standard* 

There was a great deal of pressure from both military 
and commercial users to have an open architecture for 
modular instruments. The U,S Air Force asked the XLA.TE 
User's Group> formed in iy84» to develop recommendations 
for the standardization of instruments on cards and ex- 
pressed a desire to use as much commercial equipment as 
possible. Out of this activity came the need tor a standard 
for all instrument manufacturers. 

CorrsortJum Formed 

in July 1987, a group of five instrument manufacturers 
committed to the development of a modular instrument 
standard based on the VMEbus standard. The original five 
are Colorado Data Systems Corporation. Hewlett-Packard 
Company. Racal Dana Instruments Corporation, Tektronix 
Corporation, and Wavetek Corporation. Other companies 
have since joined this consortium including Briiel & Kj^r 
Corporation. National Instruments Corporation, Keitbley 
Instruments Corporation, )ohn Fluke Manufacturing Com- 
pany, and CenRad Corporation. 

The major limitations of the original VMEbus standard 
for inslrument manufacturers were insufficient board space 



Standard 
VME 






Addttiof^af 
Sizes 



■i 



Module 
Sizes 

10 X 16 cm 
(3.9 X 6.3 in] 



23.3 X 16 cm 
(9.2 X 6,3 in) 



23.3 X 34 cm 
(9.2 X 13.4 in) 

36.7 X 34 crfi 
(14,4 X 13.4 ffi) 



Slot 
Spacing 

2 cm 
(0.8 in) 

2 cm 
(0.8 En) 



3 cm 
(1.2 if>) 

3 cm 
(1.2 in) 



VX I Additions 

Mieclianical: 
Card Sizes 
Compatibittty 
Cooling 

Electrical: 
Connectors 
Triggering 
Clocks 

PowerEMC; 
Voltages 
Radiation 

System: 
Autoconfigif ration 
Self- Test 
Message Passing 



Fig. 1. The VXtbus standard (VMEbus Extensions for in- 
strumentation) includes the two onginai VMEbus modufe 
sizes. AandB, plus two new larger modules, C andD Smaller 
modules can be inserted into mainframes designed for the 
larger sizes. The C siie mainframe is expected to be the most 
paputar for test systems. 
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and the lack of an adequate connector definition for in- 
strumentation. Space between modules was restricted to 
0.8 incfi. The VXIbus standard [VMEbus Extensions for 
Instrumentation] adds two new larger module sizes to the 
two module siz;es set up under the VMEbus standard. With 
the new modules, the space between modules is increased 
to 1-2 inches to permit adequate shielding and cooling. 
The VMEbus was developed for computers and did not 
define a backplane specifically for instrumentation needs. 
The VMEbus standard also did not address the problems 
of electromagnetic interference (EMI), power dissipation, 
or chassis cooling, and left some connector pins undefined. 
The new VXIbus standard covers these items as well as 
other issues such a.s protocols between modules^ configura- 
tions » memory allocations, and commands. 

As mentioned before, the general concept of the VXIbus 
standard is an open architecture for modular electronic 
instrumentation thai allows a variety of manufacturers to 
supply modules to operate together in the same mainframe 
chassis. The intention is for tlie standard to be open to 
anyone who washes to use it w^hether they are among the 
original founders or not. In keeping with its objective, the 
VXIbus standard is in the public domain. There are no 
copyrights on the documentation, and there are no patents 
or licensing requirements. Manufacturer ID numbers are 
provided free from the VXIbus consortium, and over 70 
manufacturers have requested and been granted numbers. 

The idea of VXIbus instrumentation is not to replace 
traditional instruments but ratht^r to offer distinct advan- 
tages for certain applications, including: 

■ High-speed communications between modules 
• Multichannel data acquisition 

■ Precision timing betw^een modules 

■ Smaller size for a typical instrument system 

■ Knse of integrating a test system, 

The VXIbus Standard 

There are four basic module sizes including the original 
two sizes that were part of the VMEbus standard. The.se 
original sizes are renamed A and B in the VXIbus standard. 
The larger two sizes, C (123 in^) and D (193 in^h can include 
additional connectors, as shown in Fig. 1. All the connec- 
tors are 96-pin DIN-lype and are referred to as PI, P2, and 
P3. The A size board has the PI connector. Sizes B and C 
may have the P2 connector as well as the required PI con- 
nector. The largest module size, D* may have the P3 connec- 
tor in addition to the PI and P2 connectors. The idea is to 
provide improved capability as a function of increased size. 

A VXIbus mainframe may accept up to a dozen C or D 
size modules in addition to a required Slot module. For 
very complex products, a module may span more than one 
VXIbus slot. 

To ensure compatibility, all pins on all three connectors 
are defined by the VXIbus standard. The Pi and P2 pin 
definitions are shown in Fig. 2. Their functional use and 
protocol are called out^ Also defined are the interfacing of 
a module to the backplane and the electrical characteristics 
of the backplane. The capabilities of each connector can 
be viewed as a pyramid, w^ith Pi covering the \^IEbus 
specification. P2 adds capability required for instrumenta- 
tion, such as more ground pins and more power supply 
pins. A 10-MHz clock and ECL and TTL trigger lines are 
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Fig. 2. Pin definitions for the P1 and P2 connectors. The PI 

connector fotfows the VME bus assignnients The P2 assign- 
ments shown are for slots 1 through 72. 
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also defmed on P2. Tbe P3 connector adds even more capa- 
bility' for specialized applications, including a lOO-MHz 
clock similar to the lO-MHz clock on P2. a 100-MHz s\ti- 
chronizing sigtiaL and ECL trigger lines In a star configura- 
Hon for module-to-module triggering- 

Stot Module 

To ensure that e\^er\' raainframe can perform ils 
minimum functions, a Slot Q module is defined. For X^XIbus 
systems with ihe P2 connector; the only Slot module 
requirement is to provide VMEbus system controller func- 
tions, a 10-MHz. 100-ppm system clock, and module ID 
drivers. For systems vv ifh the P3 connector, the Slot mod- 
ule must provide a 100-MHz clock. It is expected that man- 
ufacturers will add considerable more capability to the Slot 
module than the standard requires. 

Many Slot modules will be message-based devices with 
commander capability in addition to the minimum furir:- 
tions. These will be able to identify the devices in the 
system, configure resources, manage self -test and diagnos- 
tics, configure address maps, configure the commander-ser- 
vant hierarchies, and perform initial system (Operation. The 
capability of the Slot module will usually be combined 
with the VXIbus to HP-IB [IEEE 488, lEC 625] interfat:e 
function. 

The 10-MHz clock originates in the Slot module and 
is buffered on the backplane. It is distributed to each mod- 
ule as a single-source, differential ECL signal. The objective 
is to provide a high level of isolation and a common preci- 
sion time base. Typical performance deiivers less than 25 
ps of jitter. 



The ECL and TTL trigger lines defined on the P2 connec- 
tor are bused to all modules including the Slot Q module, 
as sho%vTi in Fig, 3, Any module may send or re<::eive triggers 
over these lines, and the lines selected are user- programma- 
ble. Several protocois are used to ensure proper trig^ring 
betw^een various manufacturers' modules. For example, a 
synchronous trigger protocol is defined to allow for trigger- 
ing from one module to another. Other protocols include 
handshake responses from the receiving module. The ECL 
trigger lines can deliver trigger rates in excess of 50 NlHz 
for high-performance applications exceeding the TTL trig- 
ger Imes' 12-MHz capabiUty* 

System Requirements 

It is a requirement that airflow and power supply capa- 
bility be fully specified for the mainframe. A typical main- 
frame is shown in Fig. 4. Likewise, the necessary degree 
of cooling and power requirements must be defined for the 
modules. This allows the user to match the capabilities of 
the mainframe w- it h the requirements of the modules. This 
type of inlarmation must be included in the product specifi- 
cations, and the user will be able to determine in advance 
whether ornot certain modules will work in a given main- 
frame. 

The close proximity of motlules within a mainframe 
makes EMI compatibility a necessary pari of the specifica- 
tion. For this reason, tight electromagnetic compatibility 
and noise requirements are imposed on module manufac- 
turers. In some cases, modules will have to be completely 
enclosed within a shield and grounded through the back- 
[jhioe. The requirements cover both neur-field radiallfin 
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and susceptibility and conducted radiation and suscepti- 
bility of the power supply pins. 

The VXIbus standard does allow for flexibility in the 
area of local bus communication. A local bus is one that 
runs from one module to its neighbor, but no farther. The 
local bus is intended for communication between modules 
either within a mulliple-slot module or within a family of 
modules. The definition is left to ihe manufacturer. This 
flexibility creates a new problem, that of protectino of elec- 
trically incompatible modules accidentally ronfigured into 
adjacent slots by the user. A protection scheme using 
mechanical keying provides six logical classes of keys: TTL, 
ECL, analog low [ — 5.5V to + 5.5V)t analog medium ( — 16V 
to +I6V), analog high (-'42V to -h42V). and a reser\^ed 
class. 

Along with hardware, the VXIbus standard also covers 
configuration and communication protocols. To avoid con- 
flicts between modules, manufacturers must maintain com- 
mon protocols. However the VXIbus standard does not 
define things such as the operating system ^ system hierar- 
chy, type of microprocessor, or type of network. 

The VXIbus standard specifies device prolocols so that 
nonconflicting portions of the VMEbus address space are 
used. A device will usually be a single module. However, 
several devices may exist on a single module and a single 
device may take up multiple slots. As many as 13 modules 
may exist in any one VXIbus subsystem, including the Slot 
U module- Up to 256 devices may exist in one VXIbus 
systemt which may include several mainframes and several 
Slot module.s. 

Types of Devices 

The lowest level of capability is a set of configuration 
registers accessible on PI . These registers allow the system 
to identify the device type, modeL manufacturer^ address 
space, and memory requirements. Modules with this 



minimum capability are called reglst er-ba.se d devices. All 
VXIbus devices must have these registers in the upper 16K 
of the 64K A16 address space. Each device is granted 64 
bytes in this space, sufficient for many devices. There are 
a number of registers including ID, device type, status/con- 
trol, and memory offset, The remaining register space is 
device dependent and may Ijr used for communication 
purposes. 

Memory requirements for devices needing additional ad- 
dress space must be readable in a defined register in the 
AIG address space. A resource manager reads this value 
shortly after power-on, then assigns the requested memory 
space by wTi ting the module's new address into the device's 
offset register. This method positions a device's additional 
memory space in the A24 or A32 address space. 

Instead of register-based, a VXIbus may be message- 
lias e<h Message-based devices have a higher level of com- 
munication capability than register-based devices; they 
communicate using a word-serial protocol. Generally, mes- 
sage-based devices include a microprocessor and execute 
ASCII commands. They have communication registers ac- 
cessible to other modules in Ihe system. Other types of 
modules include memory devices and extended devices. 
A RAM nr ROM card is an example of a memory device. 

Control hierarchy is also defined in a VXIbus system. A 
commander is a module able to ioitiate communication 
with its servants based on the servants' capabilities. A ser- 
vant may be either register-based or message-based. Com- 
mands to a message-based device can be sent using word- 
.serial protocol. For register-based devices, communication 
is by register reads and WTites and is device dependent. 

The VXIbus concept of a comma nd-servant relationship 
allows the creation of a virtual instrument whose collective 
capabilities yield a given measurement function. For exam- 
ple, a waveform generator might be teamed with a digitizer 
to form a network analyzer. A particular measurement 




Removable front 
card guides allow 
easy insertion of 
B^size cards. 



MountiifYg brackets 
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them to be 
Inserted easily^ 



Fig, 4, A typical C $iZB main- 
frame, showing C size cards, 
guides for 8 sue cards, and air 
flow. 
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capability is created through a combination of softv\*are and 
hardware. The eiemenls to maJce up a particuiar functional 
capability are not necessarily in the same physical location 
within the system. A vinual instniment can be created 
within a mainframe filled with modules having a variety 
of basic measurement functions and rearranged to work 
together through software to create complex measurement 
functions. 

The VXl bus standard offers a wide range of capabilities 
and coramuntcation protocols for manufacturers. However, 
these are invisible to the test system user. The user simply 
sets the logical address of each module and plugs it into 
the mainframe. 

Summary 

The VXIbus standard heralds a new type of instrumenta- 
tion thai gives users the flexibility to optimij^.e a test and 
measurement system by selecting modules from a variety 
of manufacturers, knowing that they will all work together. 
Distinct advantages over traditional rack-and-stank in- 
strumentation will also be offered ^ such as high-speed com- 
munication between modules, precise triggering, ease of 
system integration, and size reduction. 
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VXIbus Product Development Tools 

A VXIbus mainframe, a pair of modules, software, and 
accessories will help manufacturers develop VXIbus 
modules and systems more easily, 

by Kenneth Jessen 



To PROVIDE MANUFACTURERS with tools to de- 
velop VXIbus products, Hewlett-Packard has de- 
veloped a VXIbus C'Size mainframe, a Slot module, 
and VXIbus development software. Other accessories in- 
clude a breadboard module and a chassis shield. These 
tools are designed to give the VXIbus user the ability to 
develop products faster and with reduced resources. The 
list of HP VXIbus development tools iiicludes; 

■ C size mainframe 

■ C size Slot and translator module 

■ C size register-based breadboard module 

■ C size carrier for adapting A size or B size modules to 
the G size mainframe 

■ C si/*e chassis shield for EMI reduction between modules 

■ VXIbus development software for use with HP 9f)00 
Series 200 and 300 controliers 

■ VMEbus interface for HP 9000 Series 300 controllers 

• VMEbus preprocessor for HP 1650A and IfiSOOA Logic 
Analyzers. 

Selection of the C Size Module 

HP imd many other manufacturers have selected the C 
size module [9.187 by 13.:JHB inches with Pi and P2 con- 
nectors) as their primary module size lor inslrumenlation. 
For HP, this choice was influenced by analysis of many 
existing HP modular systems, including the following 
types: data acquisition, electronicswitchingjogicanalysis* 
waveform generation, microprocessor development, and 
spectrum analysis. All of these products were designed 
independenlly over a period of time, and their board sizes 
and power capacities were selected to opHmi^e economics 
and performance. It was found that the module size chosen 
for ail of these modular systems was equivalent to C size 
or smaller. Independent analysis canfirmed that the vast 
majority of instrument functions could be provided within 
a B size or C size system, and a C size mainframe can 
so [) port both sizes, 

VXIbus Mainframe 

The HP C size mainframe [Fig. 1] uses a carefully de- 
signed 12-Iayer printed circuit board for its backplane to 
provide the best possible noi,se immunity and signal integ- 
rity. This mainframe is designed to support the smaller 
VMEbus boards (A size and B size) using a hardware adapt- 
er kit. An optional chassis shield fits betw^een the card slots 
to provide additional electromagnetic isolation between 
modules. 

For cooling, two dc fans are used. Their speed is controlled 
by cooling needs based on a measurement of ambient hilake 
air temperature* Air is delivered through a pressurized 



plenum to ensure even airflow through each module inde- 
pendent of the number or location of modules in the main- 
frame. In other words. unu,sed slots need not be blocked 
off. so easy access is retained during module development. 
The variable-speed fans dramatically reduce acoustical 
noise in bench environments while delivering adequate 
cooling in warmer environments up to 55'^C. 

VXIbus Slot Module 

The HP Slot module is aimed at operation directly from 
a VMEbus interface or VMEbus computer. This module 
includes word-serial communication registers, allowing 
communication between other VXIbus devices and com- 
puters such as HP 9000 Series 300 Computers with a VME- 
bus interface. Many other VMEbus computers will be able 
to communicate with VXIbus message-based devices using 
this module as a translator. 

The translator consists of a VXIbus register-based device 
and a VXIbus message-based device. The register- based 
device is used to monitor and control the message-based 
device's word-serial communication register. It also pro- 
vides the required Slot backplane services. Each of these 
devices is a collection of registers in tbc VXIbus memory 
space that can commumcate over the VXIbus. The benefit 
to the user of this structure is that it allow^s operation of 
the module as a message-based device from almost any 
VMEbus device. 




Fig. 1. HP VXIbus devetopment ha f aware mdudes a C size 
mainframe, a Slot modute, a regiSter-based breadboard 
module, ar^d a carriBr modu/e for sma/fer A size ^nd B size 
cards. 
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Fig, 2* HP VXfbus developmeni software runs on HP 9000 
Series 200 and 300 Computers, ft is designed to verify the 
opefalion of VXlbuB rrjoduies snd to serve as a lesrntng tool 
for ttie VXIbus standard. 

The HP Slot module also supports the synchronous 
and asynchronous [two-line handshake) trigger protocols, 
Extemal BNC connectors allow the user to use external 
triggers from traditional HP-IB [IEEE 488/IEC 625) in- 
strumentation with any of the VXIbus TTL trigger lines. 

VXIbus Breadboard Module 

HP ill so offers fi C si'/Ji rngiyter-hased breadboard module 
wiiich hicludRs a IG-hii hardware interface to Ihe VXIbus 
bac:kplane. This module is designed to allow users to con- 
struct custom assemblies with minimum effort. This mod- 
ule includes backplane buffering to the VXIbus data Hues 
arid address lines. It also supports VXIbus autoconfigu ra- 
tion, bidirectiona] data transfers, interrupts, system faih 
system status, and manufacturer's ID code. Module shield- 
ing and ejector hardware are also included. 

Software 

The HP VXIbus development software runs on HP 900fJ 
Series 200 and 300 Computers (Fig. 2), It is used primarily 
to verify the operation of a VXIbus module. It is also a 
learning tool for the VXIbus standard. The program set is 



Fig. 3. The HPm646A VMEbus Interface for HP 9000 Series 
200 and 300 Computers 

a combination of callable subroutines written in BASIC. 
The code is not protected sti that users can modify lines 
to suit their specific needs. 

The development software jmplenn^nts the VXIbus re- 
sou rce manager functions when used in conjunction with 
the Slot module and the HP 98646 A VMEbus Interface 
[Fig. 3]. The resource jnauager confif^ures the ina infra me. 
assigns memory spaces to each module, reports what mod- 
ules are in the system, and allows access to their VXIbus 
capabilities. Access routines allow the user to read and 
write to any register and implenu^nt the word-serial pro- 
tocol for nie.ssage'based devices. 

The 22 routines in the VXIbus development software 
packai;e include six for memory devices, six for message- 
based devices, and six for .system configuration. 

A VXIbus development system exerciser program allows 
interactive aci:ess to other programs included in this t>a[:k- 
age. It prompts for parameter values and generates a call 
to a particular subprogram. 
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ects, including the HP Atlas 
Compilation System, the 
. I'.rii and Control Processor 
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he is mafhed and has two children He itves m 
Bfoo*^Jjne, Massachusetts In his spare time Jack 
teaoties computer science courses a! Boston Uni^- 
versity. He also tikes jogging. 



69 ^ Object Oriented Unit TeslJng : 




Steven P. Fiedler 

I The object-oriented unit 
testing discussed «n Steve 

Fiedler s article forms part 
of his responsibilities as a 
software quality engineer. 
! In this particufar study, he 
■mplemented processes tor 
I assuring software quality in 
Clinical information sys- 
tems. Before transferring to 
(he Waltham Division of HP, he was a s>'stems sup- 
port engineer for computer products and networks 
in the Valtey Forge, Pennsylvania, saJes office He 
came to HP tn a pan-time position tn 1979. then 
joined full-time in 1 9B1 , after receiving his BS de- 
gree in computer science from West Chester Uni- 
versity Steve IS a member of the ACM He was born 
in Milwaukee, Wisconsin, is married, and has two 
children He resides in Leominster. Massachusetts. 
He pursues musical interests in his church and, 
with his wife, often sings at weddings and smalf 
gatherings, He also enjoys skiing, hiking, and 
travel. 



75 IZReltabJIfty Growth Modelit : 



Gregory A. Kruger 

t In the productivity section 
^ ' o] HP's Lake Stevens in- 
strument Division, Statis- 
tician Greg Kruger's main 
responsibiiilies include 
R&D metrics, software 
■ reliability modeling, and 
I process analysis When he 
first [Ofcned HP at the Love- 
iand Instrument Dtvision 
almost eighi years ago, Greg's assignment in- 
cluded implementing statisltcal qua lily controt 
practices in manufaciuring and training people to 
use and understand them Later he was commis- 
sioned to spread Total Quality Control pracitces 
throughout the Lake Stevens Instrument Division 
Greg was bow in Waterloo, Iowa His BS rn 
mathematics/statistics (1979) and MS in statistics 
(1981) are both from Iowa Slate University He is 
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mamsd. has two chiJdfen, and otlers much ot his 
leisure time to his duties as deacon of his church 
Greg is an avid archer, serves on The board of 
directors of the Wasiiington Stale Bowhuniers, and 
edits a newsletter on the sybiect Vocal music is 
another ot his iniefesis. 



SO.. Comparlrtg MethodDfogles 1 




William A. Rscher, Jr. 

A Bill Fischer is the coaulhof 
of the comparative study of 
sfruG lured meEhodologiee 
in this issue of tfie HP Jour- 
^ nal He <s an R&D section 

^ -, manager in charge of Intel 

and Motorola emulation 
products Previously, he 
has been an R&D project 
manager, a technical sup- 
port engineer, and a product mai1<e ting engineer. 
In the ten years before he joined HP in 1984, 8iil 
was an engineer at the Hamilton Standard Division 
Of United TechnoJog^es. designing automated test 
equipment for the space shuffle Hehofds aBSEE 
degree (1973), an MSEE degree (197B). and a 
master's degree in management (1980), all from 
Rensselaer Polytechnic Institute. He has authored 
and coauihored a number of papers and articfes, 
mainly about software project management. Sill 
was bom m Attleboro. fy!assachu setts, and Nves 
with his wife and four children in Colorado 
Springs, Coiorado He enjoys running and playing 
basketball. 



Jame« W. Jost 

As a sEaiistician at HP's 
Logic Systems Division, 
Jim Jost is responsible for 
software metrics and pro- 
cess Improvements, He 
collaborated on the com- 
paraltve study of structured 
methodologies discussed 
m this issue of the HP Jour- 
nal Before he joined HP in 
1 984. he spent eight years doing breeder nuclear 
reactor research, particularly m the field of fuels. 
Jim holds a BA degree in chemistry from Tabor Col- 





lege (1970) and an MS degree m statistics (rom 
Oregon State Uniiversity (1976). He was bom in 

Hfllsboro. Kansas, and Ifves in Colorado Springs, 
Colorado, He is married and has three children . His 
favorite off-hours activities are basketball, sknng. 
and reading biographies. 



B^^ Objecl-Oriented Systems Analysis - 

Barry D. Kurtz 

Barry Kurtz authored the 
^ _^^^^^H ^^^ methodology and 
f^^^^j^^^^ acted as a fechnjcai con- 
L ^H suit ant Among other soft- 

^^^^^ ^W ware projects he has han- 
dled since he joined HP \v\ 
1976 were design of CAE 
tools lor electrical sctiema- 
tic capture and printed cir- 
cuit design and, as an R&D 
engmeer and project manager, design of [he Ma- 
terials Managementy30O0 soitware for hP 3000 
Business Computers, Barry's BSand MS degrees 
in computer science are from Bngham Voung Uni- 
versity (1987, 1988), and he is a member of both 
the ACM and the IEEE. He wast)orn in Richmond, 
Indiana, and lives in Boise, Idaho He's married and 
has a son He is active in his church and enpys 
family outings,, fishing, and amateur radio 



Teresa A. Wall 

Born in Norman, Of^ia- 
homa, Teresa Wall 
graduated Wfth a B8 de- 
gree in computer science 
from the University of 
Oldahoma in 1983. She 
joined [he Fort Collins Sys- 
tems Division of HP the 
same year. Among the proj- 
eels she f^as been workmg 
on are command groups and HP-UX systems in- 
tegration for the HP 9000 Series 300 and 500 Com- 
puters. HerconthbuEions to the OSA development 
focused on the analysis methodoiogy and toolset, 
Teresa's main professional interests are analysis 
and design methodoJogies and object-oriented 
languages. She lives in Santa Clara, Calif ornta. 




Oonria Ho 

Donna Ho is a software 

development engineer at 
HP's Software Engineering 
Systems Division On the 
OSA project, she was re- 
sponsible tor the analysis 
methodology, tool pro- 
totype, and documenta- 
tion When she came to HP 
in 1 985, she joi r>ed Corpo- 
rate Engineering to work on software development. 
Previous profess ion ai expe r i en c e in c lud es work on 
DG-UX command groups at Data General. Donna 
attended Duke University, graduating in 1985 with 
a BS degree in computer scJence,''psychology. S)^? 
was born in Honolulu. Hawaii, and lives in Santa 
Clara, California. 




91 VXfbus fntereonnactlon Standard : 



Kenneth Jessen 

Authors biography appears elsewhere in this 
section 



96 ^ VXIbliS ToOfS ; 




Kertneth Jessen 

Ken Jessen is a manufac- 
turing development en- 
gineer associated with new 
products and the develop^ 
ment of new processes for 
HP's Manufacturing Test 
Division He pined HP in 
1965 and, among various 
assignmenis, has held 
positions as service man- 
ager and distribution manager Dunng the past 
nineteen years, he has written many technical arti- 
cles describng HP products lor a variety of trade 
(ournals, some of them translated into foreign lan- 
guages (or publication abroad He has published 
four books on Colorado history and contributed to 
two technical books. Kens BSEE degree (1962) 
and MBA degree f 1 964) are both from the Univer- 
sity of Utah, He was born in Evanston, Illinois, is 
married, and has three cfuldren. He lives in Love- 
land, Colorado. His hobbies are writing, hiking, ski- 
ing, and railroad history. 
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